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Dear Scott Beliveau, 

Please find attached the search results for 09668465. I used the search strategy we discussed. 
I searched the standard Dialog files, IEEE,, Proquest, and the internet. 



If you would like a re-focus please let me know. 



Thank you. 
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File EBusiness & Industry(R) Jul/1994-2006/Feb 27 

(c) 2006 The Gale Group 
File 15:ABI/Inform(R) 1971-2006/Feb 27 

(c) 2006 ProQuest Inf o&Learning 
File 16: Gale Group PROMT (R) 1990-2006/Feb 28 

(c) 2006 The Gale Group 
File 20: Dialog Global Reporter 1997-2006/Feb 28 

(c) 2006 Dialog 
File 47:Gale Group Magazine DB(TM) 1959-2006/Feb 27 

(c) 2006 The Gale group 
File 75:TGG Management Contents (R) 86-2006/Feb W3 

(c) 2006 The Gale Group 
File 80:TGG Aerospace/Def . Mkts (R) 1982-2006/Feb 27 

(c) 2006 The Gale Group 
File 88:Gale Group Business A.R.T.S. 1976-2006/Feb 21 

(c) 2006 The Gale Group 
File 98:General Sci Abs 1984-2004 /Dec 

(c) 2005 The HW Wilson Co. 
File 112:UBM Industry News 1998-2004 /Jan 27 

(c) 2004 United Business Media 
File 141:Readers Guide 1983-2004 /Dec 

(c) 2005 The HW Wilson Co 
File 148:Gale Group Trade & Industry DB 1976-2006/Feb 27 

(c)2006 The Gale Group 
File 160:Gale Group PROMT (R) 1972-1989 

(c) 1999 The Gale Group 
File 275:Gale Group Computer DB{TM) 1983-2006/Feb 27 

(c) 2006 The Gale Group 
File 264: DIALOG Defense Newsletters 1989-2006/Feb 27 

(c) 2006 Dialog 
File 484 : Periodical Abs Plustext 1986-2006/Feb W3 

(c) 2006 ProQuest 
File 553:Wilson Bus. Abs. 1982-2004/Dec 

(c) 2005 The HW Wilson Co 
File 570: Gale Group MARS (R) 1984-2006/Feb 27 

(c) 2006 The Gale Group 
File 608:KR/T Bus. News. 1992-2006/Feb 28 

(c)2006 Knight Ridder/Tribune Bus News 
File 620:EIU: Viewswire 2005/Oct 19 

(c) 2005 Economist Intelligence Unit 
File 613: PR Newswire 1999-2006/Feb 28 

(c) 2006 PR Newswire Association Inc 
File 621:Gale Group New Prod. Annou . (R) 1985-2006/Feb 27 

(c) 2006 The Gale Group 
File 623:Business Week 1985-2006/Feb 27 

(c) 2006 The McGraw-Hill Companies Inc 
File 624 :McGraw-Hill Publications 1985-2006/Feb 27 

(c) 2006 McGraw-Hill Co. Inc 
File 634: San Jose Mercury Jun 1985-2006/Feb 26 

(c) 2006 San Jose Mercury News 
File 635:Business Dateline (R) 1985-2006/Feb 27 

(c) 2006 ProQuest Inf o&Learning 
File 636:Gale Group Newsletter DB(TM) 1987-2006/Feb 27 

(c) 2006 The Gale Group 
File 647: CMP Computer Fulltext 1988-2006/Mar Wl 

(c) 2006 CMP Media, LLC 
File 696: DIALOG Telecom. Newsletters 1995-2006/Feb 27 

(c) 2006 Dialog 
File 674:Computer News Fulltext 1989-2005/Oct W2 

(c) 2005 IDG Communications 
File 810:Business Wire 1986-1999/Feb 28 



(c) 1999 Business Wire 
File 813: PR Newswire 1987-1999/Apr 30 

(c) 1999 PR Newswire Association Inc 
File 587: Jane's Def ense&Aerospace 2006/Feb W3 

(c) 2006 Jane's Information Group 

Set Items Description 

51 863360 (TELEVISION OR TV) (3N) (PROGRAM? OR GUIDE? OR NAVIGATION?) 

52 42082 EPG OR ELECTRONIC { 3N) (GUIDE? OR SELECTIONS OR MENUS) 

53 49587 CLUSTER? (3N) (ANALYSIS? OR ANALY?ING OR EVALUAT? OR ALGORIT- 

HM? OR STATISTIC? OR SYSTEM?) 

54 432148 (SHOWS OR PROGRAMS OR MOVIES OR FILMS) (3N) (SUGGEST? OR REC- 

OMMEND? OR PROVID? OR MATCH? OR SIMILAR? OR COMMON?) 

55 12621025 CHARACTERISTIC? OR FACTOR?? OR ATTRIBUT? OR DESCRIPTION? 

56 115319 (USER OR SUBSCRIBER? OR VIEWER?? OR PERSON OR CUSTOMER?) (3- 

N) (MOOD OR EMOTION? OR FEELINGS OR STATE ( 3N) MIND OR HAPPY OR - 
SAD OR MAD OR ANGRY OR ANXIOUS OR FEARFUL OR DRAMATIC OR RELA- 
XING) 

57 172955 (SETS OR STATISTICAL (3N) ATTRIBUTE? OR CLUSTER? OR GROUPING? 

OR GROUP??) (5N)S5 

58 3492 (CUSTOMI? OR PERSONAL?) (3N) S4 

59 3230 AU= ( ZHANG, D? OR WONG, W? OR PROKOPENKO, M? OR ISLAM, F? OR 

ZHANG D? OR WONG W? OR PROKOPENKO M? OR ISLAM F?) 

510 1288 AU=(KOWALCZYK, R? OR OLDFIELD, M? OR BUTLER, M? OR TRAYERS, 

P? OR KOWALCZYK R? OR OLDFIELD M? OR BUTLER M? OR TRAYERS P?) 

511 2361 S5(5N)S4 

512 4894 (CHOICE? OR AVAILABL? OR FUTURE { 3N) SHOWING OR QUEUE?) (5N)S4 

513 0 S11(S)S6 

514 2 S12(S)S6 

515 1 RD S14 (unique items) 

516 0 S15 NOT FOODS 

517 0 (S9 OR S10) (S) (S3 OR S7) 

518 4 (S9 OR S10) AND (SI OR S2) 

519 3 RD S18 (unique items) 

520 3 S19 NOT S14 

521 3 RD S20 (unique items) 

522 0 (SI OR S2) (S) (S3 OR S7) (S)S6 

523 225 (SI OR S2) (S) (S3 OR S7) 

524 11 S23(S) (MOOD OR EMOTION? OR FEELINGS OR STATE ( 3N) MIND OR HA- 

PPY OR SAD OR MAD OR ANGRY OR ANXIOUS OR FEARFUL OR DRAMATIC - 
OR RELAXING) 

525 3 S24 NOT PY=>2000 

526 3 RD S25 (unique items) 

527 3 S26 NOT (S19 OR S14) 

528 225 S23 (S) (S3 OR S7) 

529 10 S28(S)S4 

530 10 S29 NOT (S26 OR S19 OR S14) 

531 5 RD S30 (unique items) 

532 128 S4(S)S6 

533 6 S32 (S) (SI OR S2) 

534 6 S33 NOT (S29 OR S26 OR S19 OR S14) 

535 5 RD S34 (unique items) 



21/3 ,K/1 (Item 1 from file: 88) 

DIALOG (R) File 88: Gale Group Business A.R.T.S. 
(c) 2006 The Gale Group. All rts. reserv. 

04074917 SUPPLIER NUMBER: 18827054 

Pieces of a dream, (selected editorials dealing with racial issues) 

Brewer, Caroline; Wilkins, Roger; Del Omo, Frank; Woo, William; Yellow 
Bird, Doreen; Butler, Mary Ellen ; Tucker, Cynthia; Garcia, Arnold Jr.; 
Trahant, Mark; Cose, Ellis; Farley-Villalobos , Robbie; Wong, William 
The Masthead, v48, n3, p58(10) 
Fall, 1996 

ISSN: 0025-5122 LANGUAGE: English RECORD TYPE: Fulltext; Abstract 

WORD COUNT: 64 61 LINE COUNT: 004 93 

. . . Butler, Mary Ellen . . . 

. . . Wong, William 

... of the El Paso Association of Hispanic Journalists. She is co-host 

of a monthly TV program titled "Border Journalists, " and also hosts a 
radio program titled "State of the Arts," which... 



21/3, K/2 (Item 1 from file: 148) 

DIALOG (R) File 148: Gale Group Trade & Industry DB 
(c)2006 The Gale Group. All rts. reserv. 



15441203 SUPPLIER NUMBER: 95681467 (USE FORMAT 7 OR 9 FOR FULL TEXT) 

Free space, (tech toys) . 

Lynch, Joan; Butler, Maura ; Miller, Matt 
EDN, 47, 27, 81(1) 
Dec 12, 2002 

ISSN: 0012-7515 LANGUAGE: English RECORD TYPE: Fulltext 

WORD COUNT: 107 LINE COUNT: 00011 



. . . Butler, Maura 



TEXT: 

...disk. Best of all, unlike PVRs such as TiVo, there's no fee for the 
EPG ( electronic -program- guide ) service. RCA/ Thorns on, 



21/3, K/3 (Item 1 from file: 275) 

DIALOG (R) File 275: Gale Group Computer DB(TM) 
(c) 2006 The Gale Group. All rts. reserv. 



01888780 SUPPLIER NUMBER: 17990970 (USE FORMAT 7 OR 9 FOR FULL TEXT) 

Video for the masses. (Intel's ProShare Video System 200 videoconferencing 
system) (includes a related article summarizing the review) (Software 
Review) (Evaluation) 
Wong, William 
LAN Magazine, vll, n2, pl37(4) 
Feb, 1996 

DOCUMENT TYPE: Evaluation ISSN: 1069-5621 LANGUAGE: English 

RECORD TYPE: Fulltext; Abstract 

WORD COUNT: 3904 LINE COUNT: 00303 



Wong , William 

sized screen and surround-sound audio used on the bridge of the 

Enterprise in the television program Star Trek. In fact, because of the 

trade-offs in cost and performance over an. . . 



27/3 ,K/1 (Item 1 from file: 148) 

DIALOG (R) File 148: Gale Group Trade & Industry DB 
(c)2006 The Gale Group. All rts. reserv. 

07674284 SUPPLIER NUMBER: 16429243 (USE FORMAT 7 OR 9 FOR FULL TEXT) 

Materials handbook. 

Ceramic Industry, vl44, nl, p57(80) 

Jan, 1995 

ISSN: 0009-0220 LANGUAGE: ENGLISH RECORD TYPE: FULLTEXT 

WORD COUNT: 121175 LINE COUNT: 10041 

... are used in electronic and aerospace applications. Other 

diamond- finished ceramic components include seals, thread guides , drawing 

dies and gyro bearing races. Many of these parts have tolerances down to 0 



27/3, K/2 (Item 1 from file: 608) 

DIALOG (R) File 608:KR/T Bus. News. 

(c)2006 Knight Ridder/Tribune Bus News. All rts. reserv. 
06666474 (USE FORMAT 7 OR 9 FOR FULLTEXT) 

Computers, Internet Face Close Scrutiny after School Shootings 

Jonathan Sidener 
Arizona Republic 
May 24, 1999 

DOCUMENT TYPE: NEWSPAPER RECORD TYPE: FULLTEXT LANGUAGE: ENGLISH 
WORD COUNT: 1530 

...TEXT: t make bombs. 

In his paper, Danielsen maintains that academic problems and the wrong peer 

group are factors that put youth at risk for violence. But the biggest 
factor is the home environment .. .measures found similar results. In 
self -reports, violent-game subjects were more likely to report feelings 
of anger. 

In another study, violent game subjects were able to read hostile words 
such. . . 

...relatively shorter times." 

Anderson said violent games add to the aggressive behavior prompted by 
violent television programs . He said there are no studies yet to see 
whether games have the same effect... 

27/3, K/3 (Item 1 from file: 636) 

DIALOG (R) File 636: Gale Group Newsletter DB(TM) 
(c) 2006 The Gale Group. All rts. reserv. 

04134277 Supplier Number: 54267468 (USE FORMAT 7 FOR FULLTEXT) 
VIDEO NOTES. 

Video Week, v20, nl3, pNA 
March 29, 1999 

Language: English Record Type: Fulltext 
Document Type: Newsletter; Trade 
Word Count: 1262 

(USE FORMAT 7 FOR FULLTEXT) 
TEXT: 

...weeks ago. Clients include CBS/Fox Video, Fox Lorber, Columbia House 
Video Library. All are " happy " with change, he said. Founded in 1986, 



L. A. -based BGH now has 35 employees. 



...at box office. Post-street date ad campaign will include ads in People, 
Entertainment Weekly, TV Guide , Playboy, ESPN magazine. Meanwhile, PHV 
is promoting March 30 direct-to- sellthrough release of The... 

...Hard Rock Cafe, Warner Home Video, Continental Airlines, Healthy Choice 
Popcorn, Princess Cruises, Visa, Max Factor , Warner Music Group . 
Streaming video and sound clips to promote movies, videogames and music 
sold online are available. . . 

9 



31/3, K/l (Item 1 from file: 16) 

DIALOG (R) File 16: Gale Group PROMT (R) 

(c) 2006 The Gale Group. All rts. reserv. 

10083159 Supplier Number: 86436226 (USE FORMAT 7 FOR FULLTEXT) 
Identifying viewer segments for television programs. 

Kim, Choong-Ryuhn 

Journal of Advertising Research, v42, nl, p51(16) 
Jan, 2002 

Language: English Record Type: Fulltext 
Document Type: Magazine/ Journal; Refereed; Trade 
Word Count: 7 8 60 

87), Weekend Dramas (.93), Movies (.94), Sports (.93), and Variety 
Shows (1.00) and information- providing programs such as News (1.00), 
Documentaries (.92), Current Specials (.99), Music/ Culture /Arts (1.00... 

. . .also shows the proportion of each segment viewers who include program 
alternatives in their viewing sets . 

Characteristics of viewer segments. Television viewing is a 
behavior that past research has shown to have... 

31/3, K/2 (Item 1 from file: 20) 

DIALOG (R) File 20: Dialog Global Reporter 
(c) 2006 Dialog. All rts. reserv. 

42743783 (USE FORMAT 7 OR 9 FOR FULLTEXT) 

MeeVee Unveils Video Search and Recommendation Engine Boasting Unmatched 
Intelligence and Personalization 

PR NEWSWIRE (US) 
June 06, 2005 

JOURNAL CODE: WPRU LANGUAGE: English RECORD TYPE: FULLTEXT 
WORD COUNT: 685 

(USE FORMAT 7 OR 9 FOR FULLTEXT) 

and customizable experience with icons and colors used sparingly to 
maintain the best experience; * Personalized Recommendations 
Consumers rate shows as they use MeeVee and get personalized 
recommendations for other programs they may like based on those 
recommendations; and * Personal Planner — Once consumers find a favorite 

. . . and even emails the Planner to them on a weekly basis to create a 
personal TV - guide . How it Works 

MeeVee personalizes the TV search and discovery experience with three 
primary technologies . . . 

31/3 ,K/3 (Item 1 from file: 484) 

DIALOG (R) File 4 84 : Periodical Abs Plustext 
(c) 2006 ProQuest. All rts. reserv. 

04178873 (USE FORMAT 7 OR 9 FOR FULLTEXT) 

Age identification, social identity gratifications, and television viewing 

Harwood, Jake 

Journal of Broadcasting & Electronic Media (GBEM) , v43 nl, pl23-136, p. 14 
Winter 1999 

ISSN: 0883-8151 JOURNAL CODE: GBEM 



DOCUMENT TYPE: Feature 

LANGUAGE: English RECORD TYPE: Fulltext; Abstract 

WORD COUNT: 6330 



TEXT: 

Watch Regularly) . Factor analysis of show preference data failed to 
yield a usable solution. However, cluster analysis clearly indicated 
four clusters of shows. The first consisted of 24 shows including 
situation comedies (Fired Up, Naked Truth... 

...for shows featuring younger people, participants were exposed to a 
number of short descriptions of shows similar to the descriptions found 
in TV Guide . In a set of 20 such descriptions, 12 were manipulated so 
as to feature either. . . 



31/3, K/4 (Item 1 from file: 613) 

DIALOG (R) File 613: PR Newswire 

(c) 2006 PR Newswire Association Inc. All rts. reserv. 

0001694221 I864D9F20D69711D9895AEF0ACD270B5D (USE FORMAT 7 FOR FULLTEXT) 
MeeVee Unveils Video Search and Recommendation Engine Boasting Unmatched 
Intelligence and Personalization MeeVee Empowers Consumers With Smarter, 
Easier Way to Navigate the Rapidly Expanding Channel Universe 

PR Newswire 

Monday, June 6, 2005 T12:30:00Z 

JOURNAL CODE: PR LANGUAGE: ENGLISH RECORD TYPE: FULLTEXT 
DOCUMENT TYPE: NEWSWIRE 
WORD COUNT: 68 9 

. . .want to see . " 

The MeeVee.com Service MeeVee provides a full-featured service to navigate 
TV* programming : * Sophisticated Search — Consumers can find relevant 
results for searches on show titles, keywords, cast members, or other show 
characteristics . Items are also grouped to help viewing and advanced 
options provide even more power to filter searches; * Powerful and. . . 

...and customizable experience with icons and colors used sparingly to 
maintain the best experience; * Personalized Recommendations — Consumers 
rate shows as they use MeeVee and get personalized recommendations for 
other programs they may like based on those recommendations; and * 
Personal Planner — Once consumers find a favorite... 

...and even emails the Planner to them on a weekly basis to create a 
personal TV guide . How it Works 

MeeVee personalizes the TV search and discovery experience with three 
primary technologies... 

31/3, K/5 (Item 1 from file: 813) 

DIALOG (R) File 8 13: PR Newswire 

(c) 1999 PR Newswire Association Inc. All rts. reserv. 
1148084 NYTH061 

Jim Ramo Joins TVN Entertainment as President and Chief Operating Officer 



DATE: September 4, 1997 



09:30 EDT 



WORD COUNT: 806 



.alongside companies and financial investors throughout the world. 



TVN Entertainment Corporation is a leading satellite television 
pay-per- view programming provider , delivering hit movies to 840,000 
direct-to-home (DTH) subscribers in a NVOD format, as well as... 

. . . has developed and is now bringing to market a new "end-to-end" digital 
satellite television programming and delivery system, primarily 

targeted at the small to medium sized cable operators, as well as non- 
clustered systems of large MSO's, as the economically competitive 
answer to alternate digital entertainment delivery technologies... 

9 



35/3,K/l (Item 1 from file: 20) 

DIALOG (R) File 20: Dialog Global Reporter 
(c) 2006 Dialog. All rts. reserv. 

38319183 (USE FORMAT 7 OR 9 FOR FULLTEXT) 

Alcatel to Showcase 'Video over Everything 1 Solutions at USTA's Telecom '04 

PR NEWSWIRE (US) 
October 11, 2004 

JOURNAL CODE: WPRU LANGUAGE: English RECORD TYPE: FULLTEXT 
WORD COUNT: 84 9 

(USE FORMAT 7 OR 9 FOR FULLTEXT) 

differentiation over other video service providers and enables the 
flexibility to deliver video applications and programs that provide a 
unique service provider brand experience into subscribers 1 homes. — Video 
Over Copper — The Alcatel Advanced. . . 



35/3 ,K/2 (Item 1 from file: 75) 

DIALOG (R) File 75:TGG Management Contents (R) 
(c) 2006 The Gale Group. All rts. reserv. 

00152748 SUPPLIER NUMBER: 12097244 (USE FORMAT 7 FOR FULL TEXT) 

Feeling and liking responses to television programs: an examination of two 
explanations for media-context effects. 

Murry, John P., Jr.; Lastovicka, John L.; Singh, Surendra N. 
Journal of Consumer Research, vl8, n4, p441(ll) 
March, 1992 

ISSN: 0093-5301 LANGUAGE: English RECORD TYPE: Fulltext; Abstract 

WORD COUNT: 7157 LINE COUNT: 00732 

evoke negative feelings that are normally considered unpleasant are 
frequently well liked. 

The popularity of programs eliciting negative feelings suggests 
that viewers respond differently to the negative feelings evoked by 
dramatic television programming compated with negative feelings 
elicited by real-life events. If viewers of a news program... 

...for themselves, then it follows that they would not enjoy the 
television-viewing experience. However, viewers of dramatic programs 
are aware that the program is simply "art" imitating life and not real life 



35/3, K/3 (Item 1 from file: 88) 

DIALOG (R) File 88:Gale Group Business A.R.T.S. 
(c) 2006 The Gale Group. All rts. reserv. 

06695276 SUPPLIER NUMBER: 113645917 

Development and features of a TV navigation system. (Author Abstract) 

Isobe, Tadashi; Fujiwara, Masao; Kaneta, Hiroyuki; Uratani, Noriyoshi; 
Morita, Toshiya 

IEEE Transactions on Consumer Electronics, 49, 4, 1035(8) 
Nov, 2003 

DOCUMENT TYPE: Author Abstract ISSN: 0098-3063 LANGUAGE: English 

RECORD TYPE: Abstract 



AUTHOR ABSTRACT: A TV navigation system has been developed to make it 



easier for viewers to watch television in a multi-channel era. It enables 
viewers to watch programs recommended by the system as well as to 
select programs using a personalized electronic program guide ( EPG ) 
and retrieve programs with a personal filter. The system combines a 
mechanism for recommending programs in accordance with the preferences 
and mood of the viewer with personalized program sorting and retrieving 
in an easy-to-use format. Subjective evaluation tests were held to evaluate 
the impressions of general viewers with regard to operating this TV 
navigation system and benefiting from its use. It was found that program 
recommendations were helpful and. . . 



35/3, K/4 (Item 1 from file: 484) 

DIALOG (R) File 484 : Periodical Abs Plustext 
(c) 2006 ProQuest. All rts . reserv. 

06912022 SUPPLIER NUMBER: 732807261 (USE FORMAT 7 OR 9 FOR FULLTEXT) 

Shooting People: Adventures in Reality TV 

Vinson, Jacquelyn 

Velvet Light Trap (VLLT) , n54, p80-82, p. 3 
Fall 2004 

JOURNAL CODE: VLLT 

DOCUMENT TYPE: Book Review-Mixed 

LANGUAGE: English RECORD TYPE: Fulltext; Abstract 

WORD COUNT: 1820 

TEXT: 

... in 1982 and which the authors categorize as "docusoap." 

The term reality TV first found common usage with shows such as 
Cops, but Brenton and Cohen argue that the docusoap was its prototype. 
Unfortunately. . . 

...in its thirteenth season, Real World has an important place in the 
history of reality television programming , yet, oddly enough, the 
authors gloss over it. Producers Mary Ellis Bunim andjon Murray both... 

...aesthetic influenced the production of Real World, but the show is 
mainly interested in arousing viewer emotion and addressing their 
fantasies with titillating and salacious glimpses into the lives of real 
people . . . 

...made its debut in 1992. Cops was another important event in the history 
of reality television programming because it was ridiculously cheap to 
produce by industry standards, it was very popular, and... 

35/3, K/5 (Item 1 from file: 613) 

DIALOG (R) File 613: PR Newswire 

(c) 2006 PR Newswire Association Inc. All rts. reserv. 

0001636163 I7F8E8DF01B9B11D9B0F0E436B27639B6 (USE FORMAT 7 FOR FULLTEXT) 
Alcatel to Showcase 'Video over Everything 1 Solutions at USTA's Telecom 1 04 
End-to-End Video Demonstration Focuses on the Many Ways Carriers Can Offer 
Revenue- Generating Video Services - Enabling the Final Piece of the ' Triple 
Play 1 

PR Newswire 

Monday, October 11, 2004 T15:08:00Z 

JOURNAL CODE: PR LANGUAGE: ENGLISH RECORD TYPE: FULLTEXT 
DOCUMENT TYPE: NEWSWIRE 
WORD COUNT: 853 



...differentiation. These include services such as: broadcast TV; parental 
control and content search; high-performance electronic guides ; 
Video-on-Demand and personalized local content such as news, weather, 
events, and movie listings. It's all about sharing images and emotions , 
whatever the user 1 s device (TV, PC, laptop, PDA or mobile handset). — 
Video Middleware — The Open Media Platform. . . 

...differentiation over other video service providers and enables the 
flexibility to deliver video applications and programs that provide a 
unique service provider brand experience into subscribers 1 homes. — Video 
Over Copper -- The Alcatel Advanced. . . 

7 



File 2:INSPEC 1898-2006/Feb W3 

(c) 2006 Institution of Electrical Engineers 
File 6:NTIS 1964-2006/Feb Wl 

(c) 2006 NTIS, Intl Cpyrght All Rights Res 
File 8:Ei Compendex (R) 1 970-2006/Feb W3 

(c) 2006 Elsevier Eng. Info. Inc. 
File 34 :SciSearch(R) Cited Ref Sci 1990-2006/Feb W3 

(c) 2006 Inst for Sci Info 
File 35: Dissertation Abs Online 1861-2006/Feb 

(c) 2006 ProQuest Inf o&Learning 
File 56: Computer and Information Systems Abstracts 1966-2006/Feb 

(c) 2006 CSA. 

File. 57 : Electronics & Communications Abstracts 1966-2006/Feb 

(c) 2006 CSA. 
File 65: Inside Conferences 1993-2006/Feb W4 

(c) 2006 BLDSC all rts. reserv. 
File 94: JICST-EPlus 1985-2006/Dec Wl 

(c)2006 Japan Science and Tech Corp(JST) 
File 95:TEME-Technology & Management 1989-2006/Feb W4 

(c) 2006 FIZ TECHNIK 
File 99:Wilson Appl . Sci & Tech Abs 1983-2006/ Jan 

(c) 2006 The HW Wilson Co. 
File 144:Pascal 1973-2006/Feb Wl 

(c) 2006 INIST/CNRS 
File 239:Mathsci 1940-2006/Apr 

(c) 2006 American Mathematical Society 
File 434 : SciSearch (R) Cited Ref Sci 1974-198 9/Dec 

(c) 1998 Inst for Sci Info 
File 583:Gale Group Globalbase (TM) 198 6-2002/Dec 13 

(c) 2002 The Gale Group 
File 603:Newspaper Abstracts 1984-1988 

(c)2001 ProQuest Inf o&Learning 
File 483:Newspaper Abs Daily 198 6-2006/Feb 24 

(c) 2006 ProQuest Inf o&Learning 
File 248:PIRA 1975-2006/ Jan W5 

(c) 2006 Pira International 



Set Items Description 

51 146651 (TELEVISION OR TV) (3N) (PROGRAM? OR GUIDE? OR NAVIGATION?) 

52 4778 EPG OR ELECTRONIC ( 3N) (GUIDE? OR SELECTIONS OR MENUS) 

53 118265 CLUSTER? (3N) (ANALYSIS? OR ANALY?ING OR EVALUAT? OR ALGORIT- 

HM? OR STATISTIC? OR SYSTEM?) 

54 84176 (SHOWS OR PROGRAMS OR MOVIES OR FILMS) (3N) (SUGGEST? OR REC- 

OMMEND? OR PROVID? OR MATCH? OR SIMILAR? OR COMMON?) 

55 9979782 CHARACTERISTIC? OR FACTOR?? OR ATTRIBUT? OR DESCRIPTION? 

56 7739 (USER OR SUBSCRIBER? OR VIEWER?? OR PERSON OR CUSTOMER?) (3- 

N) (MOOD OR EMOTION? OR FEELINGS OR STATE ( 3N ) MIND OR HEALTH OR 
HAPPY OR SAD OR MAD OR ANGRY OR ANXIOUS OR FEARFUL OR DRAMATIC 
OR RELAXING) 

57 1173940 (SETS OR STATISTICAL ( 3N) ATTRIBUTE? OR CLUSTER? OR GROUPING? 

OR GROUP??) AND S5 

58 113 (CUSTOMI? OR PERSONAL? )( 3N) S4 

59 30608 AU= (ZHANG, D? OR WONG, W? OR PROKOPENKO, M? OR ISLAM, F? OR 

ZHANG D? OR WONG W? OR PROKOPENKO M? OR ISLAM F?) 

510 5242 AU=(KOWALCZYK, R? OR OLDFIELD, M? OR BUTLER, M? OR TRAYERS, 

P? OR KOWALCZYK R? OR OLDFIELD M? OR BUTLER M? OR TRAYERS P?) 

511 18528 S5 AND S4 

512 5988 (CHOICE? OR AVAILABL? OR FUTURE ( 3N) SHOWING OR QUEUE?) AND - 

S4 

513 28 (SI OR S2) AND S3 

514 1 S13 AND S6 



S15 


4 


S4 AND S5 AND S6 




S16 
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S15 NOT S13 




S17 


4 


RD S16 (unique items) 




S18 


27 


S13 NOT (S14 OR S15) 




S19 


22 


S18 NOT PY=>2000 




S20 


19 


RD S19 (unique items) 




S21 


14 


S20 NOT (DISTANCE () LEARNING OR DRUG OR DRUGS OR PSAS OR A- 




LASKA OR CCD) 




S22 


44 


(S9 OR S10) AND S4 




S23 


11 


S22 AND S5 




S24 
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S23 NOT PY=>2000 




S25 
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RD S24 (unique items) 




S26 


0 


S25 NOT (DRUG OR DRUGS 


OR CONFESSOR OR FERRITIN OR CDNA) 


S27 


352 


S6 AND S7 




S28 
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S27 AND (S8 OR S12) 




S29 


3 


S27 AND (SI OR S2) 




S30 


1 


S29 NOT (S13 OR S14 OR 


S15) 


S31 
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S30 NOT COMMERCIALS 




S32 


0 


S12 AND S6 




S33 
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S3 AND S4 AND S6 




S34 


4 


S7 AND S8 




S35 


4 


S34 NOT (S29 OR S13 OR 


S14 OR S15) 


S36 


2 


S35 NOT PY=>2000 




S37 


2 


RD S36 (unique items) 




S38 


0 


S37 NOT (SPREAD () SHEET 


OR TRAINING) 


S39 


20 


S3 AND S6 




S40 


19 


S39 NOT (S34 OR S29 OR 


S13 OR S14 OR S15) 


S41 


9 


S40 NOT PY=>2000 




S42 


9 


RD S41 (unique items) 




S43 


1 


S42 NOT (HEALTH OR MATURITY OR HEAD () INJURED OR MANUFACTUR- 



ING OR HEALTH () CARE OR HEALTH () SERVICE? OR LEGAL OR VICTIM? OR 
DYADIC) 
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TELEVISION PROGRAMMING EVALUATION: A NEED-GRATIFICATION MODEL FOR 
AUDIENCE SEGMENTATION 

...of widening the range of programming and information services 
available to the American consumer. Marketing television programming is 
not much different from marketing other products. It includes research, 
advertising, comprehensive and systematic planning. Understanding viewer 
preferences in television programming is essential for the successful 
marketing of cable and network programming directly to viewers. 
Empirical . . . 

...escape from problems, etc.). The basic premise of this study is that the 
choice of television programming varies according to the need 
gratification expected from watching television (i.e., reasons for viewing, 
enjoyment derived, etc.). 

A study was conducted to determine viewers 1 choice patterns of 
television programming . A sample of 139 women was shown videotapes of 20 

television program segments and asked to relate their perception, 
evaluation, categorization and choice of each show on scales provided in 
the questionnaire. A hierarchical clustering algorithm then was used to 
group respondents according to their choice of all 20 programs; it yielded 
three distinct groups (segments) of television viewers. The analysis of 
these clusters was based on a need satisfaction paradigm that identifies 
three types of needs associated with... 

...Cluster 1 were identified as "television embracers," the segment that is 
the most accepting of television and its programming . In the need 
gratification context, embracers watch more "escapist" shows than the other 
segments and watch mainly to get away from their problems . This is an 
emotional orientation toward television. 

Viewers in Cluster 3, identified as "television protesters, " are 
the most selective in choosing programming, and. . . 
...as education and income among clusters. 

Grouping audience members according to their choice of 20 
television programs has proved to be a useful approach to segmentation. 
This approach shows not only behavioral... 

. . .more useful and enjoyable medium, and understanding audience needs more 
fully by direct assessment of television programming . 



17/3, K/l (Item 1 from file: 34) 

DIALOG (R) File 34 : SciSearch (R) Cited Ref Sci 
(c) 2006 Inst for Sci Info. All rts. reserv. 

05040811 Genuine Article#: TK931 No. References: 44 
Title: PSYCHOSOCIAL FACTORS IN MATERNAL PHENYLKETONURIA - WOMENS 
ADHERENCE TO MEDICAL RECOMMENDATIONS 

Author (s): WAISBREN SE; HAMILTON BD; STJAMES PJ; SHILOH S; LEVY HL 
Corporate Source: CHILDRENS HOSP,BIOCHEM GENET UNIT, IDA SMITH BLDG, 300 

LONGWOOD AVE/BOSTON//MA/02115; HARVARD UNIV, SCH MED, DEPT 

PSYCHIAT/BOSTON/ /MA/021 15 ; HARVARD UNIV, SCH MED, DEPT 

NEUROL/BOSTON//MA/02115; UNIV MASSACHUSETTS, DEPT 

PSYCHOL/BOSTON//MA/02125; TEL AVIV UNIV, DEPT PSYCHOL/IL-69978 TEL 
AVIV/ /ISRAEL/ 

Journal: AMERICAN JOURNAL OF PUBLIC HEALTH, 1995, V85, N12 (DEC), P 
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Language: ENGLISH Document Type: ARTICLE {Abstract Available) 

Title: PSYCHOSOCIAL FACTORS IN MATERNAL PHENYLKETONURIA - WOMENS 
ADHERENCE TO MEDICAL RECOMMENDATIONS 

Abstract: Objectives. This study identified factors predicting adherence 
to medical recommendations in maternal phenylketonuria which can result 
in severe fetal damage... 

. . .pregnancy. 

Conclusions. Women with phenylketonuria differed from their 
acquaintances and diabetic women in many respects, suggesting that 
special programs are needed. Greater emphasis on reproductive 
decision making is especially needed. Interventions that focus on. . . 
Research Fronts: 94-0414 002 (PLANNED BEHAVIOR; PHYSICAL-ACTIVITY 

INTENTION; USER ATTITUDE; SUBJECTIVE NORM; HEALTH PSYCHOLOGY) 
94-4339 001 {SELF-ESTEEM OF ADOLESCENTS; DEAF PEOPLE; SOCIAL 

COMPETENCE) 
94-7371 001. . . 
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Two essays related to labor market behavior: Demand for publicly provided 

job training programs in Illinois and poor health, asset accumulation, 
and early retirement behavior 

...for public job training programs in Illinois. My first objective is 
to determine which personal characteristics influence support for an 
increase in spending on publicly provided job training programs for the 



unemployed across various income groups in Illinois. In particular, I 
empirically test for. . . 

...of chronic health conditions on asset accumulation and retirement. 
Compared to a healthy worker, a person with a chronic health condition 
throughout his or her working life may have had reduced labor force 
participation and. . . 
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...provide customer energy services. Three aspects of an ESCO 
operational planning are emphasized, i.e., relaxing customer demand 
using load management programs, serving customer energy services using 
energy contracts purchased through a... 

. . . and management . 

The major contributions of this research include: (1) classifying 
and categorizing the factors influencing an ESCO operation at various 
operational levels; (2) developing various mathematical models to... 

...load management programs; (3) showing why and how an ESCO may 
utilize load management programs to provide energy services to the 
customers; (4) developing mathematical models to show how an ESCO... 
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...televised agricultural information programs (TAIP) . The objectives 
of the study focused on demographic and background characteristics of the 
farm operators; their viewership potential; size of the audience; farm 
viewer preferences; feelings about TV as a source of agricultural 
information; reasons for watching (or not watching) such. . . 

...was that programs were televised at inconvenient times which affected 
their level of viewing such programs . They suggested weekday evening 
scheduling of farm programs, grain and livestock market information on the 
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...Abstract: video scene analysis. As an example, we consider the problem 
of discriminating five types of TV programs , namely commercials, 
basketball games, football games, news reports, and weather forecasts. A 
set of low. . . 

. . . The linear separability of different classes under the proposed feature 
space is examined using a clustering analysis . The effective features 
are identified by evaluating the intracluster and intercluster scattering 
matrices of the... 

...these features, a neural net classifier was successful in separating the 
above five types of TV programs . By evaluating the changes between the 
feature vectors of adjacent clips, we also can identify. . . 
...Identifiers: clustering analysis ; 
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Summary of the 'ThinkAbout' Cluster Evaluation : Collection 

Information 

...the 3-program 'Collecting Information' cluster of ThinkAbout, a series 
of 60 15-minute instructional television programs for fifth and sixth 
graders designed to strengthen reasoning skills and to review and reinforce 
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'ThinkAbout' Cluster Evaluation : Collecting Information 

...the 3-program 'Collecting Information' cluster of ThinkAbout, a series 
of 60 15-minute instructional television programs for fifth and sixth 
graders designed to strengthen reasoning skills and review and reinforce 
language. . . 
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...Abstract: video scene analysis. As an example, we consider the 
problem of discriminating five types of TV programs , namely 
commercials, basketball games, football games, news reports, and weather 
forecasts. A set of low... 

...The linear separability of different classes under the proposed feature 
space is examined using a clustering analysis . The effective features 
are identified by evaluating the intracluster and intercluster scattering 
matrices of the... 

...these features, a neural net classifier was successful in separating 
the above five types of TV programs . By evaluating the changes between 
the feature vectors of adjacent clips, we also can identify. . . 



21/3, K/5 (Item 2 from file: 8) 

DIALOG (R) File 8 : Ei Compendex(R) 

(c) 2006 Elsevier Eng. Info. Inc. All rts. reserv. 

04277115 E.I. No: EIP9511290927 1 

Title: Model for the selection process of the design modification 



guideline (application to packaging and assembly design of electronic 
systems) 
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...Abstract: we made a prototype of a design modification guideline 
selection support system. To make a system , modification guidelines were 
clustered by multivariate analysis . We used a decision table consisting 
of clustered level knowledge for the prototype. The synthetic... 

Identifiers: Skill; Process knowledge; Association; Multivariate analysis 
; Unanticipated function; Electronic system; Design modification 
guidelines 
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Title: SIMULATION ET EVALUATION D 1 UNE GRILLE DE PROGRAMMES POUR LES 
CANAUX DE LA TELEVISION FRANCAISE. 

Author: Roucairol, C; Jacquet-Lagreze, E.; Bouroche, J. M.; Martinet, M. 
Corporate Source: Univ Paris 6, Paris, Fr 

Conference Title: Operational Research T 84, Proceedings of the Tenth 
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...Abstract: whole grid, depending on the data used and the population 
chosen (i. e. after a cluster analysis ). In each case, it is possible 
to optimize a grid using quadratic assignment programming. The... 

Identifiers: FRENCH NATIONAL TELEVISION SYSTEM; QUADRATIC ASSIGNMENT 
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...communications research, there is no one to one correspondence 
between the reasons audience members watch television and the program 
choices which they make. The central objective of this study was to conduct 
a multivariate... 

...canonical correlation analysis. 

The study took the form of an informal three-way canonical correlation 
analysis by first clustering 2,476 subjects on a battery of 139 leisure 
interests and activities using Ward f s... 
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TELEVISION AUDIENCE — PROGRAM CONTENT INTERACTION IN MEANING 
CONSTRUCTION: THE CUING PROCESS 

...actual program content is skeletal. 

To identify the cues and related knowledge in a fictional television 
program , 54 adults individually viewed a police drama, and they were 
asked to stop the program. . . 

...but interesting differences between study conditions. 

Five types of subjects were identified using non-hierarchical cluster 
analysis of their impressions of the detective. Types did differ in how 
responsive they were to... 
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...of Minnesota interviewing course. Each student interviewed a 
stranger, obtaining the respondent's opinion of television programming . 
The students audio-taped and transcribed the interview. These transcripts 
constituted the data for the. . . 



...Frequency counts of category occurrence were the basic procedure for 
describing the interviews. Chi-square analysis and cluster analysis 



were also used to answer the research questions. 
As a control, t-tests were used. . . 

...questions; and (8) probes, a rarely used message, seemed to elicit 
personal and emotional responses. 

Cluster analysis revealed four interviewer methods, each 
distinguished by the messages interviewers most often used in the... 
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. . .VIEWING TO SHOPPING ORIENTATIONS AND LIFE STYLES: THE EXAMINATION OF 
PERCEPTUAL AND PREFERENCE DIMENSIONS OF TELEVISION PROGRAMMING 

...as these advertisements potentially reach many diverse market 
segments. However, if the audience of various television programs could 
be profiled with respect to such useful descriptors as life styles and 
shopping orientations. . . 

...be more efficiently matched to the audience. 

Toward this end, previous researchers have investigated whether 
television programs can be classified into program types based on a 
single attribute — the most salient feature the programs have in common. 
Consumer choices among television programs may, however, be based on a 
multiplicity of attributes requiring a multidimensional approach which 
would. . . 

...objective of this study was to define the perceived dimensions for a set 
of ten television programs , explore the nature of the dimensions, and 
suggest which programs are perceived to be similar. . . 

...second objective of the study was to identify segments of the population 
which have similar television program preferences and investigate how 
these preference groups differ with respect to life styles, shopping 
orientations . . . 

...Research Panel using a self-administered questionnaire. The 
questionnaire contained two major sections—one obtaining television 
programming perceptions and preferences and another gathering life styles 
and shopping orientations. The demographics were chosen... 

...shopping orientation constructs were identified. 

Using Multidimensional Scaling (MDS) it was found that the ten 
television programs were perceived in three dimensions which were 
labeled masculine-feminine, humorous-serious, and realistic-unrealistic. 
The perceived television program types matched those a priorily 
hypothesized. MDS appears to be a viable technique to explore dimensions of 



television programs which may not be obvious but are nevertheless 
important and to position the programs relative to these dimensions. 

Using cluster analysis , the study provided evidence to support 
the notion that there are groups of consumers which. . . 

...previous research findings as the preference groups generally portrayed 
a preference for a set of television programs rather than a single 
program type dominating the preference group. 

Unsing Multivariate Analysis of Variance... 

...programs rather than on all of one type. Further research with a broader 
range of television programs is suggested in order to produce more 
distinctive preference groups. 
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...between samples. 2. No significant classification was obtained with 

quantification theory IV. 3. With a cluster analysis four or five 
groups were obtained. The principal factor governing this clustering 
was the use. . . 

...DESCRIPTORS: cluster analysis ; 
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Presentation method of the caption in TV programs and its effects. 

ABSTRACT: Now, various captions are used in various television programs 
. However, it is the present condition that there is no definition 
clearly carried out about . . . 
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TV Popular programs and age group. Part 2. From the audience rating 
survey in the Kan to district, conducted. . . 

...ABSTRACT: rating survey for indivisuals over 7 years old. In this paper, 
the relation between popular TV programs and age group is anlayzed 
using the age data which is included in the survey. . . 

...classified by sex and six age groups. Then it is compared yearly to 
carry out cluster analysis for the changing process and the 
characteristics of popular programs. As the result, the following... 

...DESCRIPTORS: cluster analysis ; 
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. . .partner, thinker, friend, similar to people) , "limited" (limited, 
useless, unpredictable) and servant (only one item) . Cluster analysis 
of computer-related behaviors and self rating of expertise produced five 
meaningful user types: "very... 

. . .mostly for word-processing. 

Issues of interpersonal control were not found to be related to 
feelings , images, or user types. Personal efficacy was generally related 
to positive relations to the computer. While all users... 
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Selection of a program for viewing in TV system that provides electronic 
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. . . ISLAM F F . . . 



. . . KOWALCZYK R . . . 



OLDFIELD MA... 

TRAYERS P . . . 1 / 



. . . WONG W Y . . . 



. . . ZHANG D M 



Abstract (Basic) : 

with characteristics that best match the sets and notifying the 
user of an availability of programs that best match the sets as 
program recommendations. 
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Information display for displaying electronic program guide , displays 
program table that provides information of program designated by user, 
according to setup display. . . 

Abstract (Basic) : 

. . . program table comprising cells having predetermined length for 

storing preset program information. A display controller sets up 
display conditions for displaying program contents stored in a storage 
unit (91) , so that. . . 

For displaying electronic program guide { EPG ) that 
provides information on television programs . 
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. . .Abstract (Basic) : procedure that specifies the arrangement and content 
of the file according to a set of attributed that are used to control 
the display of the pictures. The editing procedure is partitioned... 

...of which may be initiated separately for organising the file according 
to one or more attributes characteristic of that level . . . 

...level and particular picture for editing are selected, a message 

identifying the one or more attributes for the selected level is 
displayed, and indica are assigned to the attributes () for the 
selected pictures. The assigned indica are accumulated in a memory from 
which they may be recalled to display the selected pictures in the file 
according to the attributes of the level. (71pp Dwg.No.0/13) 

. . .Abstract (Equivalent) : level may be initiated separately for organizing 
the picture file according to one or more attributes characteristic 
of that level; selecting an editing level; selecting and displaying 
particular pictures for editing; displaying a message identifying said 
one or more attributes associated with the selected level; assigning 
indicia to the one or more attributes for the selected pictures; and 
storing the assigned indicia in a memory (20) from which. . . 

...be recalled to display the selected pictures in the electronic picture 
file according to said attributes of the selected level as specified 
by the stored attribute indicia. (33pp) 

. . .Abstract (Equivalent) : to the magazine, and dependent upon different 
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Fulltext Availability: 
Detailed Description 

Detailed Description 

... several forms. In one form one or more users purchase a particular 
character whose content provider provides programs or scripts which 
animate features and personalities of specific characters. 

The aforementioned modular technology 1 is ... the like). 

Hi another preferred embodiment of this invention, a toy responds to 
"currently defined mood with appropriate behavior improvement 
interaction package. 



In another preferred embodiment of this invention, a toy. 
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Fulltext Availability: 
Detailed Description 



Detailed Description 

the data retrieval mechanism calls being placed directly within the 
domain 
obj ect; 

Figure 160 shows the interrelationship between a database, a persist, 
and an account; Figure 161 illustrates that the... server system by 
including an audience which is external to the company can result in 
dramatic increases in system usage. The additional demand and increased 
usage placed on existing legacy systems... 



17/3, K/2 (Item 2 from file: 349) 

DIALOG (R) File 34 9:PCT FULLTEXT 

(c) 2006 WIPO/Univentio. All rts. reserv. 

00164699 

STEREOLITHOGRAPHIC BEAM PROFILING 
PROFILAGE DE FAISCEAU S TERE OL I THOGRAPH I QUE 

Patent Applicant /Assignee : 
3D SYSTEMS INC, 



Inventor (s) : 

SPENCE Stuart Thomas, 

TARNOFF Harry, 

ALMQUIST Thomas, 
Patent and Priority Information (Country, Number, Date) : 

Patent: WO 8911085 Al 19891116 

Application: WO 89US1559 19890417 (PCT/WO US8901559) 

Priority Application: US 88830 19880418; US 88816 19881108; US 88837 
19881108; US 88907 19881108; US 88801 19881108 
Designated States: 

(Protection type is "patent" unless otherwise stated - for applications 
prior to 2004) 
JP KR 

Publication Language: English 
Fulltext Word Count: 292227 
Fulltext Availability: 
Detailed Description 
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... of the letters significant, 

5.4 The Supervisor's Keyboard Commands 

While the Supervisor is relaxing and counting down the 
wait period, you can now take one of four actions. 



lo. 
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Detailed Description 

new system and the I 0 development environment are defined through 
this process. 

Security Standards, Guidelines and Procedures 

Security standards, guidelines and procedures provide security 
direction to the implementation. They will help define how the security 
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Detailed Description 

... can communicate with each other. Encapsulation protects the 
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Class libraries are very flexible. As programs grow more complex, more 
programmers are forced to reinvent basic solutions to basic problems over 
...and responsibilities. 

The commitments, expectations, and responsibilities are documented and 
agreed upon within the project group . Commitment may be obtained by 
negotiation, by using input and feedback, or through joint development... 
capability of the IT organization. The interview aids are used by the 
assessment team to guide them through interview sessions with 
assessment participants. 
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Advanced TV Navigation System with Easy 
Program Selection Method 

M. Fujiwara 1 ,!. Isobe 1 , N. Uratani 2 ,T. Morita 2 

1 NHK Engineering Services, Japan; 2 NHK Science & Technical Research 

Laboratories, Japan 

Abstract 

We have developed an experimental TV navigation system to assist viewers in selecting 
appropriate TV programs even when there are many digital broadcasting channels. 
This report describes a new system configuration and algorithm that can provide 
various navigation methods to suit a viewer's wishes, using an on screen electronic 
program guide with various standard style sheets at HDTV resolution. 

Introduction 

In Japan, satellite digital broadcasting began in December 2000, and terrestrial 
digital broadcasting will begin in December 2003, making HDTV and interactive 
TV a reality. With the associated increase in the number of channels and kinds of 
program, an advanced TV navigation system is needed. This should be suitable for 
various kinds of viewers, without distinction in age or sex, and should make it easy 
to select programs and watch television comfortably. We have already conducted a 
study on an advanced TV navigation system to assist viewers in selecting appropriate 
TV programs even when many channels are available. 

Recently, we have developed a new experimental TV navigation system [1]. This 
includes an innovative system configuration and an algorithm that can provide 
various navigation methods to suit the viewer's wishes. It uses an on screen electronic 
program guide, including various standard style sheets at HDTV resolution. 

In this system, the viewing habits of TV viewers are divided into four major 
categories as follows [2]: 

(a) Concentrated viewing 

(b) Viewing TV while doing something else 

(c) Diversion viewing 

(d) Diversion viewing while doing something else 

This experimental system includes a viewer profile database that stores the viewer 
interests and preferences. It also has a program information database, which stores 
broadcast schedules, that are continuously updated using service information (SI) 
data transmitted over the digital broadcasting channel, and this system is designed 
to have an interactive human interface equipped with flexible pointing devices 
suitable for various kinds of viewers. 
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Total system overview 

The system overview for advanced TV navigation system is shown in Figure 1. 
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Downloadable 
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Ex. 1) PDA type Remote 
Controller and 
Navigation Pointer 



Ex. 2) Table type 
Navigation Pointer 
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recommendations 
for you. 

OK Here it is! 




INTERESTING PROGRAMS 
for you! 




HDTV Rec. with 
Navigation System 



Figure 1: Total System Image for Advanced TV navigation System 

In this figure, the top-left corner shows a digital broadcasting station and the 
bottom-right corner shows a reception environment for digital broadcasting in the 
home, transmitting via satellite or terrestrial, including broadband Internet. Also an 
old viewer is trying to select a program with the TV navigation system. 

The expected functions of digital TV 

The NHK Broadcasting Culture Institute has recently reported about the result of a 
survey concerning modern TV viewer's attitudes in Japan. This survey was carried 
out as a memorial event for 50 years of TV in October 2002. Figure 2 shows one of the 
simplified results of the survey of users expected functions of digital TV, in order of 
the number of replies, where multiple replies were allowed. 

Looking at this data, we can find out several functions required of a digital TV-set 
and also of an advanced TV navigation system: 

1 More than half of viewers want a 'Home Video-server Function' so they can watch 
any program they want, whenever they want. 
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2 One half to one quarter of viewers 
want functions such as more- 
channels, Trogram-linked Data 
Broadcasting 7 , and HDTV - so they 
expect to be able to select their 
required program from within a 
large number of programs, with 
additional data on the requested 
programs and additional picture 
clarity of HDTV. 

3 Surprisingly, less than one fifth of 
viewers want 'Multi-angle', 'Internet 
Display' and an EPG. Perhaps, the 
reason for this result is that digital 
broadcasting has just started, and 
viewers do not yet understand the 
convenience of these functions. We 
can guess that at this time viewers 
think of a PC style interface - which 
is not easy to use, especially for older people - and think of an EPG as being the 
same as a grid-based TV listings like in a newspaper or magazine. 

From the replies to this survey, we can propose a new target for the development of 
an advanced TV navigation system for the digital broadcasting age, where viewers 
are assisted in the easy selection of a program even when many programs are being 
broadcast simultaneously, and the system collaborates in TV navigation functions to 
select programs using the 'Home Video-server' function/HDTV function, 'Internet 
Display' function and also EPG function. 

Experimental system configurations for advanced 
TV navigation 

Figure 3 shows an example block diagram of the system configuration for a multi- 
channel digital broadcasting receiver with an interactive TV navigation scheme, 
where the upper half denotes a conventional digital HDTV receiver, enabling multi- 
reception for BS (broadcasting satellite), 110E CS (communication satellite), and 
terrestrial broadcasting, and the lower half shows an additional part for an 
interactive TV navigation scheme equipped with a program information database 
that stores broadcast schedules, that is continuously updated using service 
information (SI) data transmitted over the digital broadcasting channel, and an 
interactive human interface with flexible pointing devices which are suitable for 
various kinds of viewers. 

Figure 4 also shows an example of an experimental scheme for Interactive TV 
navigation system, where two software engines - one for retrieval/sorting of the 
service information database, the other for drawing recommendations from the user 
profile database - are installed, multiplexed by the HI controller, and passed to the 
HDTV display and the simple human interface such as a navigation pointing tool to 
select items with ease to use for various viewers [3]. 
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Figure 2: Users Expected Functions of Digital TV 
(NHK Oct 2002) 



International Broadcasting Convention 



Broadcasting Services 
<Satellite> 
BSIIO'CS 
Terrestial (Cable) 



(Front End) 



(Back End) 




(Conventional Digital Receiver) 



Viewer 



( Broad Band \ ^ 





HDD (to be used also for Home Server) 



(Navigation Support Block) 
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Figure 4: An Experimental Scheme for Interactive TV Navigation System 



Viewing habits and TV navigation system 

Average TV watching time has remained around three and a half hours per day in 
Japan. The recent tendency of viewing habits can be summarized as 'Background 
TV-style viewing [2] . This is an attitude with no particular purpose to get knowledge 
or information, only to 'spend the time with easy and comfortable feeling'. Viewing 
habits including this and traditional information-oriented ones are generally 
classified as follows. 

(a) Concentrated viewing (Viewing with purpose) 

(b) Viewing TV while doing something else (Alternating between viewing while 
doing something else and viewing with purpose) 

(c) Diversion viewing (Diversion viewing with no set purpose) 

(d) Diversion viewing while doing something else ('Background TV-style viewing) 

Compared to habits (a) and (b), habits (c) and (d) can be treated as passive TV 
viewing. Such viewing habits are not fixed for any one viewer but change according 
to the time of day or viewer mood. A TV navigation system must be able to deal with 
any variety of viewing habits and with their tendency to change. 
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As shown in Figure 3 and 4, the experimental TV navigation scheme with human 
interface has multiple digital RF tuners and decoders for digital TV signals, and 
the resulting TV programs are displayed on the HDTV screen, after selection by 
the navigation process, which is also displayed on the HDTV display as an OSD 
(On Screen Display). The stages of the navigation process are given below. 

The digital TV signals contain the content descriptions such as titles, summaries, 
classifications of broadcasting programs and their broadcasting schedule as service 
information (SI) [7] [8]. The schedule for a whole week is stored in the SI database 
(SIDB). Viewer information such as age, sex and program preference are implicitly or 
explicitly stored as profile data (PFD), which also contain his or her prevalent mood. 
With reference to the SIDB and PFD, the program retrieval and sorting engine and 
the program recommendation engine selects programs and shows them on the 
display. The viewer selects the program through interactive operations and watches 
it. The following three methods of selection are available in connection with the 
foregoing four viewing habits. 

1 Retrieval: The viewer watches programs related to items of interest if available. 

2 Sorting: The viewer selects and watches a program from among those that are 
currently being broadcast. 

3 Recommendation: The viewer watches a program recommended by the system as 
optimal or selects a program from those recommended as suitable for the way the 
viewer would like to pass the time. 

Table 1 shows an outline of reference data, program sorting algorithm, the number of 
operations for each selecting method, including the number for evaluation test methods. 



Test 
methods 


Selecting 
methods 


SI data 


Profile 
data 


TV navigation system operation 


No. 
ops 


III 


I 


(1) 
Program 
Retrieval 


event name, 

text 
information 

(event 
descriptor) 


None 


Displays event names of all the 
broadcast programs, person 
names and place names included 
in the program description as 
candidate words. Retrieves 
programs with the word chosen 
by the viewer. 


>5 


(2) 
Program 
Sorting 


content 
nibble level 1, 
level 2 
(content 
descriptor), 
event name, 

text 
information 


The viewer sorts one from all the 
programs using the timetable 
displayed in layers according to 
program classification. Selection 
from a specified interest area 
adapted to the viewer is also 
possible. 


3,4 


age, sex 
and 
preference 

mood/ 
motivation 


II 


(3) 
Program 
Recom- 
mendation 


Programs adapted to the time 
zone and interest area are 
recommended. The viewer can 
also choose a program right for 
his or her prevalent mood 
(relaxing, informative, dramatic). 


0,1 



Table 1: Program selecting methods 
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Subjective evaluation tests 

Evaluation tests were conducted to examine how the users respond to the TV 
navigation system. This was intended as a rough evaluation of the program selection 
process including program recommendation. In this tests, three selection methods, 
I: retrieval/sorting, II: recommendation and III: both of retrieval/sorting and 
recommendation were given as parameters. And also, two types of TV viewing, 
am: morning and pm: evening was also set. For the evaluation tests, 30 evaluators in 
total were chosen, male and female, younger people (in their 20s and 30s) and older 
people (over 65). 

For general evaluation, the extent that they actually want to use this system was 
measured in a hundred point scale, asking about subjects of all six selection methods 
Iam-IIpm after being given the following instruction: 

'Assuming that a TV receiver having these types of program-selection methods was 
installed in your home, and that you would use the three selection methods that you 
were earlier shown in the two scenarios given, please evaluate each selection method 
in terms of "I would like to use this'' with 100 points being the highest score.' 

Users' feedback about the program selection operations using the TV navigation 
system was rated in five ranks depending on how far 24 pairs of adjectives and 
adjective verbs were considered applicable. 

The program recommendation process handles programs that are currently being 
broadcast as well as programs that will be starting shortly. Generally speaking, 
however, it is not difficult to imagine how the types of programs one desires and 
even their method of selection might change according to the scenario in which one 
watches television within the course of a day. In the morning, for example, a viewer 
might want to check out the news and the weather report for that day, while in the 
evening, when relaxing after dinner, the same viewer might like to watch a more 
dramatic or entertaining program such as a movie, drama, or live sports event. 

In this case, we assumed some criteria of personal weighting factor for preference 
mood/motivation for a user to select a program for the purpose, according to the 
time zone of a day, for example, morning ,afternoon, evening and night, dividing 
into several cases of users conditions that how they spend the time in these zones, 
adding weight based on viewer preferences. 

Table 2 shows an example of how personal preferences change according to time of 
day in terms of three program attributes: relaxing, informative, and dramatic. 



Time zone 




Morning 


Afternoon 


Evening 


Night 


How to spend 
time? 


relaxing 


20 


50 


30 


60 


informative 


70 


30 


20 


20 


dramatic 


10 


20 


50 


20 



Table 2: Examples for personal weighting factor 

Figure 5 shows an example of display information, which is flexibly designed and 
described in XML. Other related examples are shown in Figure 7 at the end of 
this paper. 
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Figure 5: An example of TV Navigation portal Menu with an HDTV Video Window, designed for young women 
and described in XML 



Results of the tests 

Figure 6 shows average values and standard deviation of the evaluation results. 
Standard deviation ranged from 10 to 16 points, and the following characteristics 
could be observed, even if a significant difference could not be said to exist. 

1 On comparing in terms of scenarios for using the TV navigation system, watching 
in the morning was found to have a high evaluation for selection procedure 
II while watching in the evening had a high evaluation for selection procedures 
I and III. 

2 For the younger age group, a difference of 10 or more evaluation points for each 
selection method could be seen between am and pm, and for the older age group, 
that difference was small. 

3 Selection procedure II (recommend) received a high evaluation in the morning 
while selection procedure III (the combination of recommend, sort, and retrieve) 
received a high evaluation in pm. 

We can interpret the above findings as saying that program recommendation as a 
guidance method of a TV navigation system is useful for a younger age group in the 
morning, a busy time in the day for such viewers. They also indicate that the combin- 
ation of sorting and retrieval is useful for an older age group at any time of day. 

To clarify how the TV navigation system was evaluated, we performed a principal 
component analysis on the evaluation by impressions for all subjects with respect to 
the 24 adjective and adverb pairs. From this evaluation, the contribution rate of the 
first principal component is 40.2%, that of the second one is 15.0% and that of the 
third one is 5.3% for a total of 60.5%, showing that the first component is understood 
as 'efficient', the second as 'rich and pleasurable' and the third as 'easy', respectively. 

The TV navigation system offers a program selection aid for digital TV 
broadcasting. The system recommends programs suited for viewers. As a result 
of subjective evaluation tests on the system, it was found that the program 
recommendation facility was highly evaluated by both younger and older people. 

We confirmed that the program recommendation engine is necessary for this kind 
navigation system, however, further improvement will be made to the program 
recommendation engine and the sorting and retrieval engine in view of the examined 
relationship between the actual operation of SI and the viewers' preference. 
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Recently, we found the similar 
articles described for 'Personal- 
ized Electronic Program Guides 
for Digital TV 7 [4] and others 
[5], [6]. Their systems are 
mainly designed to be a content 
recommendation system or 
personalized EPG system for 
digital TV, using a PC-based 
system architecture combined 
with a web-based schedule data- 
base. On the other hand, our 
navigation system is designed to 
be a new receiving system to 
assist viewers to select programs 
easily among a large number of 
digital programs according to 
viewer's wishes, using an on- 
screen electronic program guide 
with various standard style 
sheets at HDTV resolution. 

Navigation tool to assist human interface 

From the results which have already been described above, our research efforts must 
be aimed at a TV navigation system personalized to a wider range of viewers 
through a universally designed human interface. 

When we select a program, first we must select items displayed on the TV screen 
with some pointing device, such as a mouse. It would be good if we could use a 
PC style pointing device, even though the viewer may be found over a range of 
distances from the TV, for instance from 3 to 5 meters away. For this purpose, we are 
now developing two types of navigation pointer, one is a remote controller type 
navigation pointer and the other is a touch sensitive device, where viewers 
may select programs on the device in their hands as if they were touching the 
TV screen, even though it may be several meters away. An illustration of sample 
usage of these devices is shown in Figure 1 as Ex.1) PDA type Remote Controller 
and Navigation Pointer, and Ex.2) Tablet type Navigation Pointer. In these cases, 
the resultant pointing data for users to select program items are sent to the 
TV navigation system, using some wireless technologies, including small power 
radio and infrared transmission. 

Conclusions 

We have developed an advanced TV navigation system, where different kinds of 
viewers can select programs easily, and watch TV in comfort. 

This paper described the overall system overview and the concept behind our TV 
navigation system, with its newly developed algorithms and its interactive interface 



Scenes 

am: Morning (for example, when having breakfast and 
planning to go out soon afterwards) 
pm: Evening (for example, when relaxing after having dinner) 
(Note: We can compare the results of methods between am 
and pm over one day, or among am or pm). 




Iam I pm II am II pm III am III pm 
selection methods and time (am/pm) 



3 Young people 
I Old people 



I: Retrieval/Sorting 
II: Recommendation 
III: Both of I & II 



Figure 6: Result of general evaluation 



Conference Publication 2003 



for the new age of digital HDTV broadcasting, both satellite and terrestrial. When 
the TV navigation system has been realized, assistance will be provided so that users 
can select the program they want quickly and easily, using on-screen TV navigation 
guides at HDTV resolution. If equipped with a 'Home Video-server' function, viewer 
will even be able to watch their requested program several days later. 

Furthermore, our research efforts will continue, aimed at developing a TV 
navigation system which can be flexibly personalized for a wider range of viewers 
through a standard and universal human interface. 
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Figure 7: Examples of Display Information for TV Navigation 

Figure 7 (a): An example of the Program Recommendation Menu with an HDTV Video Window 
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Figure 7 (b): An example of the User's Mood Submenu in the Program Selection Menu 
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Figure 7 (c): An example of the User's Genre Submenu in the Program Selection Menu 
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Development and Features of a TV Navigation System 

Tadashi Isobe, Masao Fujiwara, Hiroyuki Kaneta, 
Noriyoshi Uratani, and Toshiya Morita 



Abstract — A TV navigation system has been developed to 
make it easier for viewers to watch television in a multi- 
channel era. It enables viewers to watch programs 
recommended by the system as well as to select programs 
using a personalized electronic program guide (EPG) and 
retrieve programs with a personal filter. The system combines 
a mechanism for recommending programs in accordance with 
the preferences and mood of the viewer with personalized 
program sorting and retrieving in an easy-to-use format 
Subjective evaluation tests were held to evaluate the 
impressions of general viewers with regard to operating this 
TV navigation system and benefiting from its use. It was found 
that program recommendations were helpful and that the 
types of programs desired by a viewer and their method of 
selection differed according to time of day. . 

Index Terms — personalized TV guides, program 
recommendation, subjective evaluation test, TV navigation 

l. Introduction 

The digitization of television is improving quality as in 
HDTV and adding functionality as in data broadcasts 
combined with programs. The dramatic progress made in 
compression-coding technology, moreover, is giving birth to a 
multi-channel era. There is true concern, however, that this 
trend toward multi-functional and multi-channel television will 
deliver more information than viewers can handle resulting in 
information overload. In such a situation, it becomes difficult 
to select a program that one would like to watch, and even if 
one knows about a program that one wants to watch, there is 
always the possibility of missing that program. The need is 
therefore felt for a TV navigation system that can eliminate 
this difficulty and worry in program selection and make it easy 
for the viewer to find and watch desired programs from the 
many and varied programs available. 

1 This work was supported in part by the TAO project 
(Telecommunication Advancement Organization of Japan); "Research and 
Development of the Next Generation Broadcasting Technology". 

T. Isobe and M. Fujiwara are with NHK Engineering Services, Inc., 
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morita.t-eg@nhk.or.jp) 



When envisioning digital television of the future, one issue 
that arises in relation to its spread and advancement is how 
viewers will go about accessing programs [1]. In Japan, this 
issue has been duly recognized, and researchers have studied 
agent-based reception environments and indexing of video 
segments as part of Integrated Services Digital Broadcasting 
(ISDB) [2], There have also been reports on IEEE related 
development of a smart electronic program guide (EPG) that 
recommends programs in accordance with a user profile [3]; 
on the development and operation of a personalized television 
listings system (PTV) that presents a personalized program 
guide via the Internet optimized by viewer reaction [4], [5]; 
and on personalized program recommendation systems using 
e-commerce techniques and new concepts in interactive 
television [6]. All of these studies on TV navigation were 
considered as performed on the premise that a viewer wishes 
to watch certain types of programs. 

Taking recent trends in television viewing habits into 
account, our aim is to make it easier for the viewer to select 
programs based on various individual viewing habits. Our 
desire, in fact, is to support even viewers that have no 
particular desire at the moment but would like to view 
something. To this end, we developed a TV navigation system 
that combines program recommendations, sorting, and 
retrieval. This system does not use the Internet; it operates on a 
standalone basis incorporated in a HDTV receiver. We held 
system trials with a focus on viewer evaluation and performed 
subjective evaluation tests. Here, with the aim of developing 
new evaluation methods as well, we also performed 
experiments on evaluating the impressions of general viewers 
and the extent to which they would like to use the system 
following their trials with the TV navigational equipment. 

2. TV Viewing Habits and Program Selection 

In Japan, the average TV viewing time is 3.5 hours per day, 
which has been the case since 1965. Viewing habits, on the 
other hand, have been changing gradually, and in 1985, a new 
trend began to take hold. Specifically, while viewers have 
always enjoyed the simultaneity and sense of presence that TV 
could offer, many of them were now turning on the TV set for 
only a vague reason at best and watching only things that 
interested them in a fragmentary manner. It has also been 
reported that recent viewing habits could be summarized as 
"viewing to pass the time" [7], This is TV viewing with no 
objective as to content or information, or in other words, "to 
pass the time in a carefree and pleasant frame of mind." All in 
all, TV viewing habits can be categorized as follows. 
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Table 1 Program selection methods 



Methods 



SI data 



Profile data 



TV navigation system operation 



No. 



Program 
retrieval 



Program 
sorting 

program 
recommen- 
dation 



event name, text 
information 
(event descriptor) 



content nibble 
level 1, level2 
(content descriptor), 
event name, text 
information 



None 



age, sex and 
preference 
mood / 
motivation 



Displays event names of all the broadcast programs, person names 5- 
and place names included in the program description as candidate 

words. Retrieves programs with the word chosen by the viewer. 

The viewer sorts one from all the programs using the timetable 3,4 
displayed in layers according to program classification. Selection 

from a specified interest area adapted to the viewer is also possible. 

Programs adapted to the time zone and interest area are 0, 1 

recommended. The viewer can also choose a program right for his or 

her prevalent mood (relaxing, informative, dramatic) 




Setting program 
by user 



^iewii 



Top page 



Time out 
Recommended PR 



Select PR 



Select EPG 



< EPG ) 



Select PR 



iewing program 



0 



Fig.l Operation of the TV navigation system 



Retrieval 



Program 
retrieval 



a. Concentrated viewing: The viewer focuses on the TV 
watching, for example, news or an educational program, or 
a movie or drama in a manner similar to watching a play at a 
theater. 

b. Viewing TV while doing something else: The viewer 
watches typhoon reports while doing housework, watches 
news or live sporting events while eating, etc. 

c. Diversion viewing: The viewer watches TV to kill time 
or to pass time in a carefree, relaxed mood. 

d. Diversion viewing while doing something else: The viewer 
does housework or eats in a room where a TV that is 
normally on acts like a room clock. 

Compared to habits (a) and (b), habits (c) and (d) can be 
treated as passive TV viewing. Such viewing habits are not 
fixed for any one viewer but change according to the time of 
day or viewer mood. A TV navigation system must be able to 
deal with any variety of viewing habits and with their tendency 
to change. 

3. Concept of a TV Navigation System 
The basic operation flow of our TV navigation system is 
shown in Fig. 1. The aim of this flow is to reduce the number 
of required operations for passive viewing habits and to deal 
appropriately with specific viewer interests. For example, a 
viewer that has just turned on the TV receiver can do nothing 
and simply watch the program currently showing on a certain 



channel if the time in question corresponds to a time zone in 
which the viewer always watches that channel. Alternatively, 
the viewer can sit back and watch a program recommended by 
the system. In addition, the viewer may elect to select a 
program. In this case, the viewer can do so from a list of 
recommendations or from an EPG that list several hours of 
programs from the present based on the viewer's personal 
profile. Finally, the viewer may also search for a program or 
programs related to a current topic of interest for up to one 
week into the future by specifying a program category and 
keywords. Program selection as described above can be 
divided into the following three types. 

(1) Program retrieval: The viewer watches programs 
related to his or her interests if available. 

(2) Program sorting as personalized EPG: The viewer 
selects and watches a program from among those that are 
currently being broadcast. 

(3) Optimal program recommendation: The viewer 
watches a program recommended by the system as optimal or 
selects a program from those recommended as suitable for the 
way the viewer would like to pass the time. 

Table 1 summarizes reference data, program selection 
algorithm, number of operations, and other information related 
to each of the above selection methods. The operation 
algorithm of the developed system is described next. 
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4. Development of a TV Navigation System 
Fig. 2 shows the configuration of the TV navigation system 
that we have developed. The system assumes a TV set that can 
receive, decode, and display digital TV signals (DTV) on an 
HDTV display and that can be equipped with the TV 
navigation function. The TV navigation system first references 
a service information database (SIDB) storing TV broadcast 
schedules and profile data (PFD) storing viewer program 
preferences. Then, on the basis of this information, it selects 
programs that have a high probability of being liked by the 
viewer using a program retrieval/sorting engine and a 
recommendation engine, and presents selection results in table 
form on the display. The viewer can now decide on what 
program to watch from this table using interactive operations. 

The DTV signal mentioned above includes content 
descriptions such as program titles, summaries, and categories 
as well as broadcast schedules as service information (SI) [8], 
[9]. One week of broadcast schedules from the present will be 
saved in the SIDB based on this SI. At the same time, the 
viewer's age, sex, and program preference will be implicitly 
and explicitly stored as PFD. 

4.1 Program retrieval/sorting engine 

The retrieval/sorting engine retrieves and sorts programs in 
accordance with the viewer's user profile and interests and 
displays the results as an EPG. The viewer selects a program 
to watch from those listed on the EPG. 

In program retrieval, how best to go about inputting search 
keywords is a major issue. Given that whatever method is 
chosen will be incorporated in the TV receiver, various 
schemes have been considered for setting search keywords 
freely. These include voice recognition and a "soft keyboard" 
for inputting characters sequentially from a keyboard image 
displayed on the screen. A method of selecting a keyword from 
a set of keywords that have been broadcast and displayed on 
the screen has also been proposed, but this scheme requires a 
keyword production and transmission system. In contrast to the 
above, our TV navigation system extracts words from the text 
making up the program titles and descriptions in the EIT and 
displays those words as keyword candidates. From this 
display, the viewer selects those keywords that best suits his or 
her current interests and initiates a program search. Needless 
to say, this is not the same as being able to set keywords freely 
since those that can be chosen have been simply taken from 
program descriptions in broadcast schedules. One advantage of 
this approach, however, is that a program corresponding to the 
keywords in question has to exist. In addition, high-speed 
searches can be performed by using a database that stores 
correspondences between keywords and programs. Fig. 3 
shows the processing flow of program retrieval. For the system 
scale assumed here, three types of keyword candidates — 
persons' names, place names, and general nouns — are 
displayed and programs for a maximum of one week from the 
present are processed. At present, about 1960 persons' names, 
590 place names, and 4510 general nouns can be extracted as 



keyword candidates from one week's worth of digital satellite 
broadcasts (76 channels) in Japan. This number of keywords 
could not, of course, be displayed on one screen at the time of 
program selection. The subjective evaluation tests described 
below were conducted with the number of keyword candidates 
kept to 100 or less by limiting the broadcast-schedule time 
period and the viewer's areas of interest. 

Fig. 4 shows the processing flow of program sorting. In 
program sorting, the engine obtains the user profile, stores it in 
PFD, and updates that data on the basis of the viewer's 
operation history. In our TV navigation system the viewer sets 
first PFD with his or her attributes such as age, sex, area of 
residence, and occupation, and then combines this data with 
favorite program categories. 
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Fig. 2 Configuration of the TV navigation system 
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Fig.3 Flow chart for program retrieval 
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Fig. 4 Flow chart for program sorting 
Next, the PF data processing of the system computes the 
viewer's degree of interest with program classifiers called 
content nibble level 1 and level2 set and then the data store in 
PFD. A viewer's PFD can change according to the time of day, 
and different PFDs can therefore be set for each time zone. In 
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the evaluation experiment described below, however, PFD 
remains constant throughout the day and the degree of interest 
can take on only two values, Tor '0\ For the next version of 
our experimental system, our plan is to divide the day into six 
time periods and assign ten levels to the degree of interest. 
Now, in relation to program broadcast schedules stored in 
SIDB, the program filtering of the system compares the 
program classifiers included in the EIT with PFD as described 
above and creates a list of programs having a high possibility 
of being liked by the viewer (i.e., personalized EPG). In the 
current version of the system, the viewer selects a program 
using this personalized EPG, which lists programs beginning 
sometime over the next two hours. 

4.2 Program recommendation engine 

In program recommendation, the system presents the viewer 
with a suitable program even when the viewer does nothing at 
all after turning on the power to the TV set. In this regard, the 
recommendation engine can select one program that it thinks 
to be optimal for the viewer or can offer several candidates for 
the viewer to choose from. The program recommendation 
process handles programs that are currently being broadcast as 
well as programs that will be starting shortly. As mentioned 
above, however, it is not difficult to imagine how the types of 
programs one desires and even their method of selection might 
change according to the scenario in which one watches 
television within the course of a day. In the morning, for 
example, a viewer might want to check out the news and the 
weather report for that day, while in the evening, when 
relaxing after dinner, the same viewer might like to watch a 
more dramatic or entertaining program such as a movie, drama, 
or live sports event. Table 2 shows an example of how 
personal preferences change according to time of day in terms 
of three program attributes: relaxing, informative, and 
dramatic. Fig. 5 shows the processing flow of the program 
recommendation engine. The program recommendation engine 
first determines the characteristics of each program that is 
currently being broadcast or soon to be broadcast by assigning 
a value to each of the above attributes based on program 
classification. It then adds a weight based on viewer 
preferences how to spend time. And computes an index value 
for recommendation indicating the degree to which the viewer 
would like to view that program in that time period as follows: 

IVR = J^PR(i)Fi(t) (1) 

IVR: Index value for recommendation 

PR(i): Program characteristics of the ith attribute 

Fi(t): Personalized weighting factor 

i: Program attribute (relaxing, informative, and dramatic) 

t: Time zone 

The program with the highest index is recommended by the 
system. At this time, the system can also present a list of 
several programs with high index values so that the viewer can 
select a program from that list if so desired. 



5. Subjective Evaluation Tests 

We examined the impressions of people that tried using our 
experimental TV navigation system. Our aim here was to make 
a basic comparison of the program selection methods 
including "program recommendation" as implemented by the 
system. These subjective evaluation tests are summarized 
below. 

A. Evaluation method 

To evaluate the impressions of people that used the TV 
navigation system and performed program-selection 
operations, we employed the 24 pairs of adjectives and 
adverbs listed in Table 3 and had subjects evaluate each pair 
on a 5-level basis. Subjects were also asked to specify the 
extent to which they would say "I would like to use this" with 
100 points being a perfect score. 

B. Subjects 

There were a total of 30 subjects divided into two groups of 
15 with subject age in the first group ranging from 20 to 30 
years old and that in the second group from 65 and up. The 
male/female ratio was about 50/50. 

C Viewing conditions 

We adopted standard viewing conditions as used in ITU-R 
TV-quality evaluations[10] and a 30-inch liquid-crystal HDTV 
display with viewing distance set to 4H. 

Table 2 Examples for personal weighting factor 



Time zone 


morning 


afternoon 


evening 


night 


How to relaxing 


20 


50 


30 


60 


spend informative 


70 


30 


20 


20 


time? dramatic 


10 


20 


50 


20 



Program 
characteristics 



Personalized 
factor weighting 



Recommended 
program list 



Fig.5 Flow chart for program recommendation 

D. Test parameters 

Subjects were asked to evaluate six different cases 
combining the following program-selection procedures (the > 
mark indicates the initial screen) and viewing scenes. 
Selection procedures 1 : > Sorting, retrieval 
2: > Recommendation 
3: > Recommendation, sorting, retrieval 
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Scenes a: Morning (for example, when having breakfast and 
planning to go out soon afterwards) 
b: Evening (for example, when relaxing after having 
dinner) 

The order of presenting the above selection procedures to 
each subject was random while that of the scenes alternated 
between "a then b" and "b then a". 

E. Contents 

Two series of the experimental contents were made for 
interactive operation of our TV navigation system. Fig. 6 
shows the initial screen of experimental content used in the 
subjective evaluation tests. 

F. Test Procedures 

The subjective evaluation tests were performed under the 
above conditions from January 10 to 27, 2003. They were 
carried out separately for each subject over seven sessions. In 
each of the first six sessions, a subject was asked to try using 
the system for one of the six combinations la-3b of procedures 
and scenes with usage time limited to 15 minutes. The subject 
was then asked to evaluate that selection method by specifying 
one of five levels for each of the 24 sets of adjectives/adverbs 
in Table 3, i.e., by indicating which side of each 
adjective/adverb set was most applicable to that selection 
method. Here, to obtain a wide evaluation of user impressions, 
we used a broad range of adjectives and adverbs. Furthermore, 
to lighten the psychological load on the subjects, we asked 
them in our pre-test instructions to "evaluate each selection 
method intuitively without giving it too much thought." After 
the above evaluation, the subject was asked to make a note for 
future reference as to what extent they would say "I would like 
to use this" for that selection method on a scale from 0 to 100. 
Finally, in the seventh session, the subject was asked to 
recollect his or her use of the TV navigation system in the 
prior six sessions and to give a general evaluation of each 
selection method in terms of "I would like to use this" again 
with 100 points being the top score. Because each of the above 
operation methods had many common elements, the time 
needed for each trial decreased gradually for many subjects. In 
these tests, instead of simply presenting the TV program that 
had been selected, both the selected program and a still screen 
indicating the name of the program were displayed in an 
overlaid manner. 



earlier shown under the two scenes stated, please evaluate each 
selection method in terms of 'I would like to use this' with 100 
points being the highest score." 




Fig.6 An example of display information 



Table 3 Adjective Pair for Evaluation Testing 



No 


Adjective Pair 


1 


Easy to use - difficult to use 


2 


Easy to select - difficult to select 


3 


Organized - disordered 


4 


Clear - unclear 


5 


Easy to understand • difficult to understand 


6 


Easy to get used to - difficult to get used to 


7 


Easy to see - difficult to see 


8 


Simple - complex 


9 


Calming - irritating 


10 


Streamlined - excessive 


11 


Easy - difficult 


12 


Relaxed - rushed 


13 


Quick - slow 


14 


Novel - outdated 


15 


Cool - unsightly 


16 


Happy - sad 


17 


Interesting - dull 


18 


Enjoyable - boring 


19 


Warm - cold 


20 


Rich - poor 


21 


Free - restricted 


22 


Light - heavy 


23 


Cheerful - gloomy 


24 


Elegant - crude 



6. RESULTS AND DESICCATIONS 
6.1 General evaluation 

Subjects were asked to make a general evaluation of all six 
selection methods la-3b after being given the following 
instruction: 

"Assuming that a TV receiver having these types of 
program-selection methods was installed in your home, and 
that you would use the three selection methods that you were 
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evaluation points could be seen between morning and 
evening, and for the older age group, that difference was small. 

(3) Selection procedure No. 2 (recommend) received a 
high evaluation in the morning while selection procedure No. 3 
(the combination of recommend, sort, and retrieve) received a 
high evaluation in the evening. 

We can interpret the above findings as saying that program 
recommendation as a guidance method of a TV navigation 
system is useful for a younger age group in the morning, a 
busy time in the day for such viewers. They also indicate that 
the combination of sorting and retrieval is useful for an older 
age group at any time of day. 

6.2 Evaluation of subject impressions 

To clarify from what points of view the TV navigation 
system was evaluated, we performed a principal component 
analysis on the evaluation by impressions for all subjects 
across sessions 1 to 6 with respect to the 24 adjective and 
adverb pairs in Table 3. Table 4 shows the top three principal 
components in terms of contribution rate and eigen coefficients 
as determined by this analysis. The contribution rate of the 
first principal component is 40.2 %, that of the second one is 
15.0 % and that of the third one is 5.3 % for a total of 60.5%. 



Table 4 Contribution rate and eigen coefficients of principal component analysis (All data) 

1st principal component 40.2% 2nd principal component 15.0% 3rd principal component 5.3% 



"Efficient" 




"Richness & pleasure" 




"Carefree" 




Simple 


0.283 


Rich 


0.408 


Relaxed 


0.306 


Streamlined 


0.268 


Free 


0.307 


Easy 


0.281 


Easy to select 


0.263 


Enjoyable 


0.269 


Cheerful 


0.258 


Easy to understand 


0.260 


Interesting 


0.264 


Simple 


0.248 


Organized 


0.252 


Cheerful 


0.218 


Organized 


0.239 


Easy to use 


0.250 


Warm 


0.212 


Warm 


0.238 


Quick 


0.246 


Novel 


0.204 


Calming 


0.238 


Clear 


0.240 
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Fig. 7 Result of general evaluation 
Fig. 7 shows average values and standard deviation of the 
evaluation results. Standard deviation ranged from 10 to 16 points, 
and the following characteristics could be observed even if a 
significant difference could not be said to exist. 

(1) On comparing in terms of scenes for using the TV 
navigation system, watching in the morning was found to have a 
high evaluation for selection procedure 2 while watching in the 
evening had a high evaluation for selection procedures 1 and 3. 

(2) For the younger age group, a difference of 10 or more 
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Fig. 8 Eigen values of principal component analysis 

From the major adjectives and adjective verbs, the first principal 
component is understood as "efficient" in use such as in simple, 
streamlined, easy to select, easy to understand, organized, and easy 
to use. The second principal component can be interpreted as 
''richness and pleasure" as in rich, free, enjoyable, and interesting, 
while the third principal component can be interpreted as 
"carefree" as in relaxed, easy and cheerful. 

On performing a principal component analysis on both the 
younger age group and older age group of subjects, no large 
difference was observed between the two for the first and 
second principal components. For the third principal 
component, however, a difference appears. In the younger age 
group, this component can be interpreted as "cheerfulness" as 
in cheerful, organized, and easy, and in the older age group, as 
"originality" as in novel, interesting, and free. In future 
research, we plan to conduct more detailed analyses such as by 
analyzing principal component scores for each selection 
procedure. 

Based on the results of principal component analysis on the 
evaluation by impressions, Fig. 8 shows average values of 
principal component eigen values for "efficient," the first 
principal component, and "richness and pleasure," the second 
principal component, over all sessions. Given that these results 
are based on the evaluation by impressions, we obtained high 
evaluations for selection methods 3a, 3b, and lb, which 
reflects about the same characteristics as the results for the 
general evaluation described earlier. The evaluation of the 
younger-age group shows the result that 3a and 3b 
corresponding to the three selection methods are efficient and 
la and lb corresponding to sorting and retrieval is pleasurable. 

Our plan is to use the above results in deciding how to 
improve the program retrieval/sorting and recommendation 
engines and to upgrade the TV navigation system. 

7. Conclusion 

This paper has described our development of a TV 
navigation system that helps viewers select programs broadcast 
by digital television. This system can recommend programs 
that suit viewer preferences, present an electronic program 
guide for program selection based on viewer preferences, and 



perform program retrievals based on viewer desires. 
Subjective evaluation tests were performed using this system 
and results revealed that program recommendation received 
high marks from both young and elderly subjects. In future 
research, we plan to carry out more detailed analyses on actual 
SI operation and viewer preferences with the aim of improving 
the program retrieval/sorting and recommendation engines of 
this TV navigation system. We also aim to introduce a human 
interface that can support a wide range of viewers as a 
"universal design" and to achieve a TV navigation system 
personalized for each major age group. We will likewise 
perform more viewer-based evaluation tests and establish new 
evaluation methods to meet the demand for easy-to-use TV 
navigation systems. 
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Abstract. Watching television tends to be a social activity. So, adaptive television needs to adapt to groups of 
users rather than to individual users. In this paper, we discuss different strategies for combining individual user 
models to adapt to groups, some of which are inspired by Social Choice Theory. In a first experiment, we 
explore how humans select a sequence of items for a group to watch, based on data about the individuals' 
preferences. The results show that humans use some of the strategies such as the Average Strategy (a.k.a. 
Additive Utilitarian), the Average Without Misery Strategy and the Least Misery Strategy, and care about 
fairness and avoiding individual misery. In a second experiment, we investigate how satisfied people believe 
they would be with sequences chosen by different strategies, and how their satisfaction corresponds with that 
predicted by a number of satisfaction functions. The results show that subjects use normalization, deduct misery, 
and use the ratings in a non-linear way. One of the satisfaction functions produced reasonable, though not 
completely correct predictions. According to our subjects, the sequences produced by five strategies give 
satisfaction to all individuals in the group. The results also show that subjects put more emphasis than expected 
on showing the best rated item to each individual (at a cost of misery for another individual), and that the ratings 
of the first and last items in the sequence are especially important. In a final experiment, we explore the 
influence viewing an item can have on the ratings of other items. This is important for deciding the order in 
which to present items. The results show an effect of both mood and topical relatedness. 

Keywords Group modeling, interactive television, social choice, adaptation, recommender 



1 Introduction 

Interactive television offers the possibility of personalized viewing experiences. Different domains have been 
identified in which this personalization would have a great impact, such as education (Masthoff & Luckin, 
2002), news (Maybury, 2001), advertising (Lekakos, Papakiriakopoulos & Chorianopoulos, 2001), and 
electronic program guides (Cotter & Smyth, 2000). Adapting television to individual viewers is a topic in itself, 
and a lot of research has already been done, particularly in the area of electronic program guides. This research 
tends to build on decades of work on content-based and social filtering. In this paper, we will explore an even 
more difficult issue: adaptation to a group of viewers. We believe this to be essential for interactive television as, 
in contrast to the use of PCs, television viewing is largely a family or social activity (Barwise & Ehrenberg, 
1988; Kasari & Nurmi, 1992). Unfortunately, television-viewing statistics do not include data on the average 
number of people watching television together and who watches television with whom (as also noted by Gillard, 
1999). It is very likely to be culturally dependent, as the number of televisions per household varies widely. 
According to a large research study in the UK (Livingstone & Bovill, 1999), television is the medium most often 
shared with family. Watching television together is top of the list of activities shared between parents and 
children, and more than two thirds of children watch their favorite programme with somebody else, nearly 
always family. Children most often watch with their siblings (Van Evra, 1998). Young people would like to 
watch television with friends, though (due to a lack of resource) many do not manage to do so (Livingstone & 
Bovill, 1999). Given the rising number of televisions in bedrooms it is likely that watching television with 
friends will be an increasingly popular activity. Already, television is the most popular conversation topic of 
young people with friends (Livingstone & Bovill, 1999). For these reasons, we believe that adaptive television 
should be able to adapt to groups of people watching together. These groups can be quite heterogeneous, and 
age, gender, intelligence, and personality influence what types of TV programmes people enjoy (Kotler, Wright 
& Huston, 2001; Gillard, 1999; Livingstone & Bovill,1999). The question then arises how one can adapt to a 



group of viewers, in such a way that each individual enjoys (and in educational programs, benefits from) the 
broadcast. 



2 Strategies for combining user models 

User modeling has been widely studied, particularly the modeling of user preferences (directly or indirectly via 
observation and inference) (see UMUAI journal, User Modeling conferences). In contrast, group modeling - 
combining individual user models to model a group- has hardly been investigated in our field. There are only 
three main adaptive systems that use it: MusicFX (McCarty & Anagnost, 1998), POLYLENS (0* Conner, Cosley, 
Konstan & Riedl, 2001), and INTRIGUE (Ardissono, Goy, Petrone, Segnan, & Torasso, 2002). MUSICFX is used 
in a company's fitness center to select background music to suit a group of people working out at any given time. 
POLYLENS is a group recommender extension of MOVIELENS, which recommends movies based on an 
individual's taste as inferred from ratings and social filtering. It allows users to create groups and ask for a 
recommendation for that group. INTRIGUE recommends places to visit for tourist groups taking into account 
characteristics of subgroups within that group (such as children and disabled). Though some exploratory 
evaluation of MusicFX and POLYLENS has taken place, for none of these systems it has been investigated how 
effective their group modeling strategies really are, and what the effect would be of using a different strategy. 
Besides, the application domains of both POLYLENS and MUSICFX differ from television viewing in the sense 
that these systems do not need to select a group of items: people normally only see one movie per evening, and 
music stations can play forever 1 . For INTRIGUE, on the other hand, it is quite likely that a tourist group would 
visit multiple attractions during their trip, but the selection of a balanced sequence has not been addressed yet. 
Our view on adaptive interactive television is that reasonably small video segments would be concatenated. The 
smaller the segments the more adaptation and real interactivity can take place. 

Though group modeling has hardly been studied in our field, the related issue of social choice (also called group 
decision making) -deciding what is best for a group given the opinions of individuals- has been studied 
extensively in economics, politics, sociology, and mathematics (see, e.g. Condorcet, 1785; Pattanaik, 1971; 
Taylor, 1995). Their construction of a social welfare function is very similar to our group modeling problem. 
Other areas in which the problem has been studied are Meta-Search, Database Middleware, Collaborative 
Filtering, and Multi-Agent systems. In Meta-Search, the ranking lists produced by multiple search engines need 
to be combined into one list (they call this the problem of rank aggregation). See, for instance, Dwork, Kumar, 
Naor, Sivakumar (2001), and Cohen, Schapire, and Singer (1999). Dwork et al. base their work on social choice 
theory, and use a variant of the method of Kemeny, which uses an extended Condorcet principle (see Section 2.2 
for an explanation of the ordinary Condorcet principle, and why we object to it). In Database Middleware, 
objects have to be ordered where each object has numerical values for multiple fields (see for instance, Fagin, 
Lotem, and Naor, forthcoming). In Collaborative Filtering, preferences of a group of individuals have to be 
aggregated to produce a predicted preference for somebody outside the group. See Pennock, Horvitz and Giles 
(2000) for an explanation of how social choice theory applies to collaborative filtering. In multi-agent systems, 
agents need to take decisions that are not only rational from an individual's point of view, but also from a social 
point of view. See Hogg and Jennings (1999) for a discussion of social rationality for agents and its links to 
Social Choice Theory. See Ephrati and Rosenschein, (1996) for how a social decision mechanism (namely the 
Clarke Tax mechanism) can be used to reach consensus between multiple non-cooperative (possibly cheating) 
agents. 

In this section, we will discuss some of the issues, and present a number of example strategies. 



2.1 Focusing our problem and introducing an example 

Assume the television has a set of items to choose from. These can be news items, quiz questions, MTV music 
clips, television programs, etc. For our discussion we will just call them items (and video clips in the 
experiments). Assume the television needs to adapt to a group of viewers. Assume the television knows who the 
viewers are, and the system has preference ratings for each of them (say from 10, really like, to 1, really hate). 



1 This would have been different if MUSICFX selected individual songs rather than radio stations. 



The problem now is which items should the television show, given that it has time for a certain number but not 
ail of them? 

An example of this situation is given below. There are three viewers, John, Adam and Mary, and the television 
has ten items to select from (A to J). For each item, it knows the preference ratings, for instance, John really likes 
A, but Adam really hates it. The problem is what should the TV show if it has time for only one item? What if it 
has time for two items? Etc. 





A 


B 


C 


D 


E 


F 


G 


H 


I 


J 


John 


10 


4 


3 


6 


10 


9 


6 


8 


10 


8 


Adam 


1 


9 


8 


9 


7 


9 


6 


9 


3 


8 


Mary 


10 


5 


2 


7 


9 


8 


5 


6 


7 


6 



This example will be used to illustrate the strategies described below, and will also be used in the experiments 
presented in Sections 3 and 4. 

Note that we have made a number of simplifications: we have assumed that a number of problems have been 
solved while actually they are still a focus of much research. For example: 

• How will the television know who is watching it? Solutions have been proposed in the form of an individual 

infrared card which registers viewers automatically (Lieberman, van Dyke, & Vivacqua, 1999), an 
individual token which you have to put on the television, a login procedure (which can take a conversational 
form, with the television asking who is watching it tonight), and a probabilistic mechanism using the time of 
day combined with a known probability of a viewer watching at that time (Goren-Bar & Glinansky, 2002). 

• How will the preferences of the individual users be determined? Social, and content-based filtering can be 

used, combined with stereotypes (see many papers in Ardissono & Buczak, 2002). Obviously, there is a 
complication in that it is difficult to make inferences from actions when a group watches the television, but 
actions at times the user watches alone could be used combined with a probabilistic model when watching in 
company. Plua and Jameson (2002) describe a mechanism by which groups of people who know each other 
well can help each other to specify their preferences. An additional complication is that an individual's 
ratings might depend on the group they are in. For instance, a teenager might be very happy to watch a 
programme with his younger siblings, but might not want to see it when with his friends. 

• Dealing with differences in rating tendencies. Not all people have the same rating behavior. Some people only 

use the ends of the scale, they either "really hate" or "really love" an item. Others only use the middle, never 
being very positive, and never being really negative. A "7" by Pete, who is always very negative, may be a 
far more positive review than a "9" by Tim, who likes everything. Note that similar differences may occur 
when ratings are inferred from viewing behavior. These differences in behavior should be taken into account 
when using ratings as input for a group model. One way to do this is to normalize them. Though we have 
not assumed the ratings of John, Adam and Mary to be completely comparable (and indeed some differences 
in rating tendencies can be seen in the example, e.g. between John and Adam), we have simplified our 
problem by giving all individuals "reasonable" rating behaviors. For instance, none of them only uses the 
ends of the scale. 

• Dealing with uncertainty. The preferences as determined by filtering mechanisms do not have to be correct 

(though accuracy is growing). For the research in this paper, we will assume the ratings to be accurate. We 
will revisit this issue in the Areas for Further Work section in the conclusions. 

• Changing groups. We assume that the group remains the same during the whole sequenced If one member of 

the group needs to leave early (a child, for instance, who has to go to bed) then it is likely that the 
preferences of that person should have greater weight when they are present. 

• Dealing with multidimensional ity. Often, you have ratings in multiple dimensions, rather than just one 

dimension. For instance, in adaptive instruction, there are many reasons for selecting an item, such as the 
student's existing knowledge (does it fulfill the prerequisites), learning goal and learning style, the 
educational flow (does it built on what has been explained before), etc. One way to match this onto the 
modeling as discussed above would be to construct a single rating for each item based on how it scores on 
these criteria. Masthoff (in press) describes a way in which the aggregation methods discussed in this paper 
can be used to aggregate ratings of different criteria. 

• How will the recommendations be presented to the group? We assume that the television decides which items 

to show, so, does not give the viewers a choice. One kind of application we are considering is a personalized 
news program. So, instead of watching a news broadcast that is the same for all viewers (like the BBC 



news), you would watch a news program with items in it that are automatically selected for the group of 
people you are watching with. Most recommender systems, in contrast, would present their list of 
recommendations to the user, and different ways have been devised on how to do this. For instance, 
Zimmerman, Parameswaran, and Kurapati (2002) discuss using celebrities (a photo combined with text) to 
present generated content recommendations (for instance, for an Electronic Program Guide). 

All of these problems merit more research, but are beyond the scope of this paper. 
2.2 Desirable properties 

In Social Choice Theory, some consensus exists about desirable properties of voting systems, but it has been 
proven that no system can have all these properties (Arrow, 1950), and, arguably, this is why different voting 
systems exist in different countries, institutions, and societies (see Cranor, 1996, for about 20 of these, and 
Section 2.3 for a selection). One analysis of 27 democracies over a period of 45 years found a staggering 70 
different voting systems being used for national elections (Lijphart, 1994). 
Some examples of desirable properties that have been proposed: 

• Pareto rule (Pareto, 1 897). 

If at least one person prefers x to y and nobody prefers y to x, then x should be above y in the ranking. If 

nobody prefers x to y and nobody prefers y to x, then x and y should share a place in the ranking. 

In our example, everybody prefers F to G. Hence, F should precede G in the group list. Similarly, F should 

precede H, as John and Mary prefer it and Adam does not mind. 

In this manner, we find: 

B should precede C D should precede B,C,G E should precede G,I 

F should precede B,C,D,G,H,J H should precede B,C,G,J J should precede C,G 

• Anonymity (May, 1952). 

If the ratings of two individuals are swapped for all alternatives, then the resulting sequence should remain 
the same. Note that dictatorship violates this rule, and that it can be well defended that in real life television 
viewing not everybody has to have equal rights all the time. For instance, if it is John's birthday than maybe 
it is fair to give John more satisfaction on that day than the others. 

• Positive A ssociation. (Arrow , 1951). 

If alternative x preceded alternative y in the sequence, then an increase in an individual's rating of x should 
maintain x's position before y in the sequence. 

If alternative x appeared equal to alternative y in the sequence, then an increase in an individual's rating of x 
should lead to x being before or equal to y in the sequence. Note that it is tempting to say that x should 
precede y in the latter case, but this is too strong a condition. For instance, if the sequence was determined 
on the basis of Plurality Voting (see below), then x and y could be equal because as many people preferred x 
to y as preferred y to x. Increasing the rating of x for one individual does not necessarily change this. 

• Condorcet winner criterion (Condorcet, 1785). 

An alternative x is a Condorcet winner if for each other alternative y: 
x is preferred to y by the majority of individuals. 

The criterion states that if a Condorcet winner exists then it should top the group list on its own. 

A weaker version allows it to share the top of the group list. 

The criterion becomes stronger when modifying the Condorcet winner definition to 

"x is preferred or equal to y by the majority of individuals." 

In our example, both E and A are Condorcet winners using this definition. 

We do not agree with this criterion (not even in its weakest form). Assume an item x is rated 10, 1, 10 (as A 
is in our example). Assume all other items are rated 9, 9, 9. Item x is then a Condorcet winner, but it can be 
argued that it should not top the group list, given the misery it produces for the second individual. 

Our resistance to the Condorcet winner criterion highlights another property we might want our voting systems 
to have: 

• Each individual 's satisfaction with the results should be above a certain threshold. 

When a sequence of items is selected for a group to watch, the individual's satisfaction could be measured at 
the end of the sequence, or, in a stronger version of this rule, at any moment in the sequence. Note that the 
latter does not necessarily mean that the individual's satisfaction with each item should be above a certain 
threshold, as satisfaction is considered in the context of the items shown so far. For instance, consider a 
sequence "8 9 3 10". If we want to measure satisfaction at any moment in the sequence, then we would 



have to measure the satisfaction after having seen "8", after having seen "8 9", "8 9 3", and "8 9 3 10". 
The satisfaction of the individual after having seen "8 9 3" could well be higher than the satisfaction with 
"3" on its own, so this sequence might pass this criterion, while a sequence "3 8 9 10" might fail. 

We will need to determine empirically whether typical TV viewers share our negative feelings about the 
Condorcet winner criterion, and whether they agree with our individual satisfaction rule. 



2.3 Example strategies 

Many strategies, also called "social choice rules", "group decision rules", and "rank aggregation functions", have 
been devised for reaching group decisions given individual opinions. We will discuss some simple ones (the first 
five originate from social choice theory and the latter five from our specific use), and illustrate them with the 
example introduced above. The example will show the 'group list' resulting from the strategy, a sequence 
indicating in which order the items would be chosen. Sometimes, two items score the same, like E and F in the 
Additive Utilitarian strategy. That is indicated in the group list by placing them between brackets. This means 
that either E is followed by F, or F followed by E. The main purpose of this section is to show that many 
different, all seemingly logical, strategies can be devised, all of which have quite distinct results when applying 
them to the example. 

1. Plurality Voting (also called 'first past the post'). Each voter votes for his or her most preferred alternative. 
The alternative with the most votes wins. This method is, for instance, used in UK elections. When a 
sequence of alternatives needs to be selected, this method can be used repetitively: first, an election is held for 
the first place in the sequence, next for the second place, etc. In the example, John would like to vote for A, E, 
or I (all ratings of 10). Adam for B, D, F, or H, and Mary for A. Traditionally in Plurality voting, each 
individual has only one vote, so, John would have to decide whether to vote for A, E, or I. If John were aware 
of the preferences of the others, then it is likely that he would vote for A, as with Mary's vote this would 
secure a majority. In our scenario, with only three individuals and ten items, it is quite likely that a vote would 
end in a tie (in contrast to politics, where the number of individuals tends to be a lot larger than the number of 
alternatives). If John were to vote for E or I, then all three individuals would vote for a different item, and 
there would be no winner. It would clearly be in John's interest to vote A. In our case, the television would 
decide on a choice for the group, and as the television would be aware of all individuals' preferences, it could 
easily accommodate strategic voting, to prevent ties. Our interpretation of Plurality Voting in this context will 
therefore be that rather than giving individuals one vote, we allow them to vote for all items that have the 
highest rating. In our example, this gives A two votes, and it becomes the start of the sequence. Next, John 
likes to vote for E or I, Adam for B, D, F, or H and Mary for E. With two votes E has most votes, and 
becomes second in the sequence. 
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Instead of using the method repetitively, each voter could vote for x alternatives (with x being the length of 
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2, Utilitarian Strategy. Utility values for each alternative (expressing the expected happiness) are used, instead 
of just using ranking information (as in plurality voting). This can be done in multiple ways: 
Additive. Ratings are added, and the larger the sum the earlier the alternative appears in the sequence. Note 
that the resulting group list will be exactly the same as when taking the average of individual ratings. For this 
reason this strategy was called the "Average strategy" in (Masthoff, 2002). This strategy (often in a weighted 



form, where weights are attached to individual ratings) is used in multi-agent systems (Hogg & Jennings, 
1999) and Collaborative filtering. This is also the strategy used in the INTRIGUE system (Ardissono et al, 
2002), with a weighting depending on the number of people in the subgroup and the subgroup's relevance 
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Multiplicative. Instead of adding the utilities, they are multiplied, and the larger the product the earlier the 
alternative appears in the sequence. 





A 


B 


c 


D 


E 


F 


G 


H 


i 


J 


John 


10 


4 


3 


6 


10 


9 


6 


8 


10 


8 


Adam 


1 


9 


8 


9 


7 


9 


6 


9 


3 


8 


Mary 


10 


5 


2 


7 


9 


8 


5 


6 


7 


6 


Group 


1100! 




148" 


mum 


mm* 


mm 


11801 


mm 


12101 





Group List: 

F E H J D I (B, G) A C 



A disadvantage of the utilitarian strategy is that an individual viewer might always lose out, because their 
opinion happens to be a minority view. This is more likely to cause problems the larger the group. After all, in 
a small group the opinion of each individual will have a large impact on the average/product. 



3. Borda Count (Borda, 1781). Points are awarded to each alternative according to its position in the individual's 
preference list: the alternative at the bottom of the list gets zero points, the next one up one point, etc. For 
instance, in our example John has the lowest rating for C, and hence, C is awarded 0 points. A problem arises 
when an individual has multiple alternatives with the same rating. We have decided to distribute the points. 
So, for example, in Mary's list B and G share the place one up from the bottom and get (l+2)/2 =11/2 points 
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4. Copeland Rule (Copeland, 1951). This is a form of majority voting. It orders the alternatives according to the 
Copeland index: the number of times an alternative beats other alternatives minus the number of times it loses 





A 


B 


C 


D 


E 


F 


G 


H 


I 


j 


A 


0 








0 








0 




B 


+ 


0 




+ 


+ 


+ 


0 


+ 


+ 


+ 


C 


+ 


+ 


0 


+ 


+ 


+ 


+ 


+ 


+ 


+ 


D 


4- 






0 


+ 


+ 




0 


0 




E 


0 








0 












F 


+ 








+ 


0 










G 


-f 


0 




+ 


+ 


+ 


0 


+ 


+ 


+ 


H 


+ 






0 


+ 


+ 




0 






I 


0 






0 


+ 


+ 






0 




J 


+ 






+ 


+ 


+ 




+ 


+ 


0 


Index 


+7 


-6, 






+8 


f£5;: 


r-61 


o- 




• ; -3- - 



Group List: 

E A F I D H J (B, G) C 



Note that in the example the resulting group list is almost identical to the one resulting from repetitive 
plurality voting. 



5. Approval Voting. Voters are allowed to vote for as many alternatives as they wish. This is intended to promote 
the election of moderate alternatives: alternatives that are not strongly disliked. This type of voting is used by 
several professional societies, like the IEEE. In our example, we could assume that John, Mary, and Adam 
vote for all alternatives with a rating above a certain threshold. They could vote for all alternatives with a 
rating higher than 5, as this means voting for all alternatives they like at least a little bit. 
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6. Least Misery Strategy. Make a new list of ratings with the minimum of the individual ratings. Items get 
selected based on their rating on that list, the higher the sooner. The idea behind this strategy is that a group is 
as happy as its least happy member. PolyLens (O' Conner, Cosley, Konstan & Riedl, 2001) uses this 
strategy, assuming groups of people going to watch a movie together tend to be small and a small group to be 
as happy as its least happy member. A disadvantage is that a minority opinion can dictate the group: if 
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7. Most Pleasure Strategy. Make a new list of ratings with the maximum of the individual ratings. Items get 
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8. Average Without Misery Strategy. Make a new list of ratings with the average of the individual ratings, but 
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MUSICFX (McCarty & Anagnost, 1998) uses a more complex version of this strategy. Their users rate all 
music stations, from +2 (really love this music) to -2 (really hate this music). These ratings are converted to 
positive numbers (by adding 2) and then squared to widen the gap between popular and less popular stations. 
An Average Without Misery strategy is used to generate a group list. To avoid starvation and always picking 
the same station, a weighted random selection is made from the top m stations of the list (m being a system 
parameter). 



9. Fairness Strategy. Top items from all individuals are selected. When items are rated equally, the others' 
opinions are taken into account. The idea behind this strategy is that it is not so bad to watch something you 
hate, as long as you get to watch the things you really love as well. This strategy is often applied when people 
try to fairly divide a set of items: one person chooses first, then another, till everybody has made one choice. 
Next, everybody chooses a second item, often starting with the person who had to choose last on the previous 
round. It continues till all items have been used. In our example, if we assume John chooses first, then John 
would like A, E, or I. He could choose E because it causes the least misery to others and has the highest 
average. Next it is Adam's turn. Adam would like B, D, F, or H. He could choose F because it has the best 
ratings for the others. Mary would choose A (her highest rating). Next, Mary would like E, which has already 



been shown, and then F, which also has already been shown. Therefore, it makes sense to let Adam choose. 
He likes B, D, or H. He chooses H, as that has the best ratings for the others. Following this strategy, we could 
end up with a group list like: E, F, A, H, I, D, B, etc. The list would, of course, be different if we let Mary or 
Adam choose first. However, we would expect A to be within the first three items, as it is the item Mary 
prefers most. 



10 Most Respected Person Strategy (Also called "Dictatorship"). The ratings of the most respected person are 
used —in our example assume that is Adam—, only taking the ratings of the others into account to choose 
between similarly rated items. The idea behind this strategy is that groups may be dominated by one person. 
For instance, some research shows that the television remote control is most often operated by the oldest male 
present. Similarly, adults may have more influence than children (could depend on the time of day, adults 
having more influence later in the day). Visitors may have more influence than inhabitants of the house. 
Special circumstances, like birthdays, illness, etc. can influence who is "the most respected" person on a 
particular moment. This strategy is used often in collaborative filtering under the name of "the nearest 
neighbor strategy": only the preferences of the individual closest in taste to the outsider are used. 
A more sophisticated use of differences in social status would be to assign weights to the individuals' ratings. 
As mentioned above, this has also been used in collaborative filtering and in the INTRIGUE system (Ardissono 
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2.4 Summary of strategies and the issue of Satisfaction 

As is clear from the summary table below, the different strategies described led to quite different results when 
applied to our example. One major difference between strategies is the emphasis placed on individual 
satisfaction, particularly avoidance of misery, compared to the satisfaction of the majority of the group. A clear 
example is the location of A in the group lists, an item that is very much hated by Adam (rating 1), but loved by 
both John and Mary (ratings 10). Plurality Voting puts A at the top of its list, and it ranks also highly in the lists 
of the Copeland, Borda, Most Pleasure and Fairness strategies. In contrast, the Average Without Misery strategy 
completely ignores A, and A also ranks at the bottom of the lists of the Least Misery and Utilitarian 
Multiplicative strategies. 

A second major difference between the strategies is whether they use only the relative position of items in each 
individual's preference list, or also the strengths of these preferences. The Plurality, Copeland, Borda, and Most 
Respected Person strategies only use relative positions, unlike the Utilitarian, Least Misery, Average Without 
Misery, and Most Pleasure strategies. 



Table 1. Summary of group lists produced by the strategies discussed above when applied to our exa mple. 




As discussed in Section 2.2, we believe strategies should have a property like "Each individual's satisfaction 
with the results should be above a certain threshold'. So, to determine how good the strategies are we need a 
way of measuring each individual's satisfaction with the sequences they produce. Note that this is something the 
choice strategies do not tell us. One way of doing this would be to have a Satisfaction Function that takes as 
input a sequence and an individual's and their friends' ratings and produces as output a number that quantifies 
the individual's satisfaction with that sequence. Ideally, such a Satisfaction Function would be empirically 
validated, for example by predicting individuals' satisfaction with sequences and then measuring in an 
experiment how satisfied they really are (see Experiment 2, Section 4). A good Satisfaction Function would be a 
fast way to test how strategies perform under many different circumstances (like group size, ratings, sequence 
length, etc.). Additionally, the construction of a good Satisfaction Function would provide valuable insights into 
what makes a good strategy. 

A basic Satisfaction Function 

In its simplest form, a Satisfaction Function would take as input a set of ratings for any sequence of clips and 
produce as output a real number. One such function that has been used a lot is Addition: the summation of the 
individual's ratings of the clips concerned. So, for instance, John's satisfaction with FEAHD would be 43 
(9+10+10+8+6), while Mary's would be 40 (8+9+10+6+7). Though without thresholds we cannot say whether 
these numbers amount to high or low satisfaction, the function would predict that John would be more satisfied 
than Mary. 

The issue of normalization 

A basic Satisfaction Function as sketched above only considers the ratings of the selected clips. It does not take 
into account how these ratings compare to those of the unselected clips. So, for instance, ratings of "6 5 6 7" 
would produce the same satisfaction whether the other clips had ratings of "9 10 10 10 10 9" or "1 2 1 3 1 4". 
To counteract this, normalization can be used, by dividing the sum of ratings of the selected clips by the 
maximal "possible" sum for that individual. For instance, the maximum sum for John for a sequence of five 
items is 47 (namely 10+10+10+9+8), while the maximum for Mary is 41. So, John's satisfaction with FEAHD 
would be 0.91 (43/47), while Mary's would be 0.98 (40/41). We could then conclude that Mary would be more 
satisfied than John (in contradiction with the results without normalization). Normalization is one way to 
counteract differences in rating tendencies. 2 

The issue of linearity 

An issue that has not been taken into account by any of the described strategies is that ratings are not necessarily 
linear. The question arises whether the difference between a "9" and a "10" should really be as big as between a 
"6" and a "7". We have the hypothesis that the further away from the middle point of the scale (in our example 
5.5 can be seen as Neutral), the larger the difference between subsequent ratings. So, both the differences 
between a 9 and a 10 and between a 1 and a 2 are larger than the difference between a 6 and a 7. To achieve this, 
we could convert the ratings "123456789 10" for our satisfaction function into " -25 —16-9—4 -1 +1 +4 +9 
+ 16 +25". This has for instance as a result that FE would give Mary a higher satisfaction (16+4=20) than JC 
(9+9=18), while the satisfaction would be equal if ratings were considered linear (FE= 9+7=16, JC = 8+8=16). 

The issue of misery 

Another question is whether satisfaction depends on pleasure only (the sum of the positive numbers as a 
proportion of the maximum achievable pleasure) or whether it is also affected negatively by disagreeable 
experiences (more than because of losing out on possible pleasure). 

The issue of order 

Until now, we have considered the Satisfaction Function to take as input a set of ratings (whether only of 
selected or also of unselected items). This assumes that the order of the sequence does not impact the 
Satisfaction. However, the impact of viewing an item on the user's happiness is likely not only to depend on the 
viewer's liking for the item in isolation, but also on the context in which the item is shown. It is well known in 
the advertising world that the context of an advertisement has an impact on its effectiveness and resulting brand 
evaluation. In particular, studies have shown that the viewer's mood (as induced by watching the preceding 



2 Note that normalization only works when we assume each individual to have some items they like. For 
instance, we would not want to conclude that a selection of "2 2 2 2" with other items of "1 1 1 1 1", would 
make this individual 100% satisfied. 



program) has a significant effect on brand evaluations (Meloy, 2000; Gardner, 1985), with the viewer responding 
more positively if they were in a more positive mood. The liking of a television program has a similar significant 
effect (Murray, Lastovicka, and Singh, 1992; Schumann and Thorson, 1990). So, we hypothesize that an item 
that is rated as, say, a 3 could be perceived more highly after having watched a 10, and less highly after having 
watched a 1 . There is also an interaction between the emotional tone of commercials and programs (Kamins, 
Marks, and Skinner, 1991): viewers preferred a sad commercial in the middle of a sad program, and a humorous 
commercial in a humorous program. This forms the basis for consistency theory, which suggests that viewers try 
to maintain a mood throughout a program. For example, after watching a "September 1 1th" news item, viewers 
might prefer watching another sad item, rather than a funny one, even when normally they would rate the funny 
one higher. Other content aspects are also likely to play a role. An item about "the position of the Kurds in Iraq" 
may be appreciated more than its individual rating suggests, after having just seen an item about "the US 
position on Iraq". 

The issue of solidarity 

Is it possible to determine an individual's satisfaction without considering the satisfaction of the others in the 
group? Would a person be as satisfied with a certain sequence when his friend got a sequence of "1 2 1 3" as he 
would be when his friend got a sequence of "7 8 6 8"? Hogg and Jennings (1999) deal with this issue in multi- 
agent systems by adding a weighted measure of the satisfaction of the society. They consider weights between 0 
(a very selfish agent) and 1 (a very sociable agent). In fact, the situation could be even more complicated: 
somebody could be jealous of the pleasure of their friend, so their satisfaction would decrease if their friend's 
ratings increased. Note that this is quite likely when siblings are watching television together. 

So, how do we decide which is the best Satisfaction Function? A way to find such function is to determine some 
plausible functions before measuring (reported) human satisfaction in an experiment, and to use these functions 
to predict the experimental outcomes. A comparison between the predictions and the real outcomes would then 
produce insights into the relative merits of the Satisfaction functions and their weaknesses. Additionally, the 
experiment could produce the needed threshold value. We will report on such an experiment in Section 4. First, 
however, we have explored how people act when confronted with the task our strategies perform. 



3 Experiment 1: How real people do it 

One can easily create hundreds of strategies (the strategies above are only the tip of the iceberg). The important 
question is which strategy is most effective and will be most liked by viewers. As a starting point, we want to 
determine what strategy real people use. We have performed a first experiment to explore this. 



3.1 Experimental design 
Method 

Subjects were divided into two groups, experiencing different experimental conditions. In both conditions, 
subjects were given the same individual ratings of three people, John, Adam, and Mary, for a set of video clips. 
In seven questions, they were asked which clips the three should view as a group, given that they only had time 
to see respectively 1, 2, 3, 4, 5, 6, or 7 clips, and why they made that selection. The task presented to both groups 
differed only in that in condition 2, "John, Mary and Adam" had been replaced by "John (29), Mary (32), and 
their grandfather Adam (81)" (see Appendix A for exact task wordings). A between-subject design was used, as 
a pilot test revealed large order effects: subjects felt compelled to change their group ratings in favor of Adam, if 
they received condition 2 after condition 1. The individual ratings had been chosen primarily to enable 
differentiating between the strategies we expected subjects to use (same ratings were used as in Section 2). In 
addition, we ensured that John and Mary had quite similar ratings, while Adam's ratings were frequently the 
opposite of the ratings of the other two. We also ensured that for one clip, namely clip A, John and Mary had 
maximal positive ratings (10), while Adam had a maximal negative rating (1). The latter would give a good idea 
of the importance subjects assigned to avoiding misery. The ratings can also be seen as representing different 
rating behavior: Adam has not used the maximum of the scale (10), while John has used it three times. 

Research questions 

We wanted answers to the following research questions: 



• Do subjects follow a clear strategy? Is it possible to describe subjects 1 individual behavior in terms of a 
logical strategy? Are the strategies discussed above being used? 

• Is there a dominant strategy? Is one strategy used by a majority of subjects, and, if so, which strategy is it? 

• Do subjects take pleasure, misery, and fairness into account? Which do they find most important? 

• Do subjects follow the rules (exhibit the desirable properties as discussed in Section 2.2)? 

• Is social standing taken into account? Does subjects' behavior change if one person in the group can be 
regarded as more important? 

We expected the results of both experimental conditions to provide some answers to the first four questions, and 
the difference between the conditions to provide some insight in the last question. Our hypothesis was that in 
Condition 2, Adam would be regarded as more important (because of his age), and the selections would be more 
geared toward his taste. Note that we have deliberately chosen to make this experiment an indirect one: rather 
than having an actual group sit down to decide what to watch, subjects were asked what they thought people 
should watch. There are two reasons for this. Firstly, we really wanted our subjects to think about what would be 
best for the group as a whole. Giving them a role to play (i.e. to represent John, Adam or Mary) could lead to 
them trying to defend their own interests (even with clear instructions to consider the group as a whole). 
Secondly, as discussed in our literature review in Section 2, individuals behave differently depending on who 
else is in the group. Some people tend to be more accommodating of others, some are more timid and others 
more outspoken, some are better at arguing their case, or are just more respected. We did not want the eloquence 
with which John, Mary and Adam argued their case to influence the outcomes. 

Subjects 

Thirty nine subjects participated in the experiment. All were final-year undergraduate students of the IT faculty 
attending a lecture of the Adaptive Interactive Systems module. The students were studying various courses 
(B.A. Computer and Information Systems, B.Sc. Computer Studies, B.Sc. Computer Science, and B.Sc. 
Software Engineering). The experiment took place in a lecture room. Subjects were assigned to experimental 
condition depending on where they sat: the left of the room was assigned to condition 1 (18 subjects, 16 male, 2 
female, average age 28, standard deviation 9.7), the right to condition 2 (21 subjects, 15 male, 6 female, average 
age 24, standard deviation 3). Students participated in the experiment voluntarily (in addition to the numbers 
mentioned above, 9 students chose not to participate). The spread over courses was similar for both conditions. 

3.2 Results and discussion 

Subjects do not seem to answer the questions independently: they responded which new clip should be added to 
the sequence they had already chosen for the previous question. This made it possible to present the results in the 
way we have done in Tables 2 and 3 (for respectively, Condition 1 and 2), only showing the new clip selected for 
each question. However, from an experimental point of view, this is not ideal: it might have influenced their 
strategy, making it perhaps less likely that they use the "fair strategy" (which only makes sense when selecting a 
larger group of clips). We need to explore whether the results would be different if we asked the subjects 
immediately to select, say, six clips. A between-subject design could be used to distinguish between different set 
sizes. 

Table 2 shows the results for Condition 1, and Table 3 for Condition 2. As can be seen in the tables, subjects did 
not always make a unique selection for a clip, sometimes they answered "D or J". We have tried to keep the 
tables as simple (and uncrowded) as possible: if a cell does not have a clip name in it, then the first name above 
it applies. For instance, subl 1 replied F to the first question. Subjects have been ordered to make the tables as 
easy to view as possible. The tables include information about how well the subjects' replies fit some of the 
strategies discussed in Section 2: 

• Bold borderlines indicate replies that are in correspondence with the Average Strategy. So, for instance, all 
replies of subl4 were the same as those by the Average Strategy. The first two replies by subl 6 were the 
same as those by the Average Strategy, but subl6's later replies differed. 

• Gray cell shading indicates replies that are in correspondence with the Least Misery Strategy. So, for 
instance, all replies of subl 1 were the same as those by the Least Misery Strategy. The first three replies by 
subl 6 were the same as those by the Least Misery Strategy, but subl6's later replies differed. 

• Bold dotted borderlines indicate replies that are in correspondence with the Average Without Misery 
Strategy. So, for instance, all replies of sub2 were the same as those by the Average Without Misery 
Strategy. 



Table 2. Results for Condition 1 (see above for meaning of shading and borderlines). 
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3 When we say something like "Average Strategy throughout", we mean that the subjects' choices are identical 
to those of this strategy. This does not necessarily mean that the subject was consciously applying this strategy. 



Note that strategies can have overlapping starts of their group lists. For instance, both the Average Strategy and 
the Least Misery Strategy allow a start of FEHDJ. This means that cells can have both a gray cell shading and a 
bold borderline. So, for instance, sub7's replies followed the Average Strategy for the first five clips, and in 
correspondence with the Least Misery Strategy for the first six clips. Also, note, we have only used the Bold 
dotted borderlines, when the Average Without Misery Strategy starts deviating from the Average Strategy. 

Do subjects follow a clear strategy? 

There is evidence that human subjects use the strategies mentioned above, particularly the Average Strategy, the 
Average Without Misery Strategy and the Least Misery Strategy. 

• Average Strategy, Two subjects in Condition 1 (subl4 and 9) and five subjects in Condition 2 (SUB8, 16, 
11, 19, and 6) exactly followed the Average Strategy. Their papers tended to show additions on them. 

• Least Misery Strategy. Three subjects in Condition 1 (subl 1, 10, and 5) and two subjects in Condition 2 
(SUB 12, and 10) exactly followed the Least Misery Strategy. Three subjects in Condition 1 (sub4, 7, 3) and 
three in Condition 2 (SUB 20, 3, 4) followed the Least Misery strategy almost completely. For clip 7, three 
selected A ("while Adam hates it, the others really like it so he will just have to put up with it", "Mary seems 
to lose out in most clips, so A is for her", "Might as well please two out of three"), three selected I 
("Closest", "At least the majority will be satisfied"). 

• Average Without Misery Strategy. Two subjects in Condition 1 (sub 1 7, 2) and three subjects in Condition 2 
(SUB2, 14, 5) exactly followed the Average Without Misery Strategy. Three subjects used a threshold of 4 
or 5 and two subjects (sub2, SUBS) used a threshold of 2 or 3. As expressed by one subject "I try to please 
all of them making sure that no is lower than five". Note that the resulting sequence for the Average Without 
Misery Strategy for a threshold of 2 or 3 coincides with Multiplicative Utilitarianism (but no signs of 
multiplications where found on the papers). 

• Fairness Strategy. Two subjects (sub6, 13) used some kind of a Fairness Strategy. Both selected A relatively 
early. They made comments like "Although Adam gave 1 mark for A, he gets to see F", "Although some 
gave some clips low marks, they all get to see some they rated highly". Other subjects applied fairness 
towards the end: "Mary 's average ratings have been low, so give her something she will enjoy" (subl, 
explaining selecting A), "Mary seems to lose out in most clips, so A is for her" (SUB3), "As Adam did not 
like A, pick D next as he scored it a 9" (SUB 1 5). 

• Approval voting. The subjects' explanations did not show any sign of using Approval voting. Nevertheless, 
thirteen subjects' sequences fall within those permitted by Approval voting with threshold 5, and eleven 
within those permitted by Approval voting with threshold 6 (seven of which are in common with threshold 
5). These seem high numbers, but it has to be taken into account that Approval voting (particularly with 
threshold 5) did not put many restrictions on the sequences it allowed. 

• Nobody used plurality voting (in either form) and nobody used the Copeland rule. Nobody completely 
followed the Borda count (though two subjects' behavior on the first four items coincides with it: sub6 and 
subl3). 

Is there a dominant strategy? 

There does not seem to be a clearly dominant strategy, but Average, Average Without Misery, and Least Misery 
are all plausible candidates for implementation. Fairness plays a role, but our human subjects did not have a clear 
strategy for applying it. 

Do subjects take misery into account? 

Many subjects take misery into account, as evidenced by the high proportion of subjects using the Least Misery 
and Average Without Misery strategies. Even subjects that do completely deviate from the Least Misery or 
Average With Least Misery strategies, like subl2, sublS, SUB21, SUB 18, SUB17, and SUB 13 avoid misery: all 
left out A and I from their selection. Therefore, preference should be given to a strategy that takes misery into 
account. 

Is social standing taken into account? 

We did not find any statistically significant differences between the conditions. Only one subject explicitly 
mentioned age as a reason for a selection: "A is not chosen because only the young ones like the topic" (SUB 5). 
Our intention of making Adam the most respected person did not completely succeed: one subject (SUB7) 
actually mentioned "Adam's scores have been ignored to some extend because of age", another (SUB21) said 
"overall pick the average highest, if there is any difference attempt to match the two people with the same age". 
Overall, it seems that this part of the experiment was not successful: in future we will have to make it more 
obvious that one person is socially more important (perhaps by making it their birthday). 



Do subjects follow the rules? 

No, subjects sometimes exhibit completely unexpected behavior. Four subjects in Condition 1 and four in 
Condition 2 selected G (ratings 6,6,5) before D (ratings 6,9,7). This seems rather illogical, and breaks the Pareto 
rule. Two of these subjects explained using disparities in ratings as a basis for selection. This would mean that a 
group is happy if everybody were equally happy or miserable. Overall, ten subjects broke the Pareto rule, in the 
following manners: 
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The only plausible explanation seems to be that subjects thought fairness to be more important than pleasure. 

Subjects also do not follow the Condorcet winner criterion. In its stronger form, A and E were both Condorcet 
winners, but almost all subjects started their sequence with F. This backs up our resistance against the Condorcet 
winner criterion. 



4 Experiment 2: How people judge the sequences produced 

In the previous experiment, we have investigated what strategies people follow and what they find important 
when making a decision on behalf of a group (for instance, misery, fairness, etc). However, the fact that, for 
instance, many people used a Least Misery strategy does not necessarily mean that our television should use this 
strategy (though it seems a reasonable option, given the results of Experiment 1). Similarly, the fact that nobody 
used the Copeland rule does not necessarily mean that the television should definitely not use that strategy. 
Perhaps that strategy was just to complex for a human to apply. In this experiment, we have turned the game 
around: instead of asking subjects to produce a satisfying sequence, we have presented subjects with sequences 
produced by the strategies, and asked them how satisfied they would be with such a sequence. We wanted to 
determine which strategy produces the most satisfaction for all members of the group. To gain a better 
understanding of what determines an individual's satisfaction, we have also compared a number of satisfaction 
functions (see Section 2.4) to see which provides the best predictions. 

4.1 Experimental design 
Method 

Subjects were told that they were going to watch video clips with their two friends. They were given the same 
individual ratings of three people, themselves, Friend 1, and Friend2, for a set of video clips. The ratings used 
were the same as those in Experiment 1, with John's ratings corresponding to their own, Adam's ratings to 
Friendl's, and Mary's ratings to Friend2's. 4 They were told the TV had selected a sequence of clips for them, 
and were asked how satisfied they and their friends would be given that sequence, and why. This was repeated 



4 We decided to use "Friend 1" and "Friend2" rather "Adam" and "Mary" to avoid any influence on the subjects' 
behavior. After all, names imply gender. It would have been better to use three male names in Experiment 1 . 
Though we did not detect any influence, we decided to avoid the risk in this experiment. 



three times: all subjects were given three different sequences on three pieces of paper, stapled together. (See 
Appendix B for the exact wording.) The table below shows the sequences used, and the reason for using them: 



Sequence 


Reason 


FEAHD 


Social choice according to Borda Count 


EAFID 


Social choice according to Copeland rule 


AEFID 


Social choice according to Plurality voting (one for one) 


FEHJDI 


Social choice according to Utilitarian multiplicative 


EFHDJB 


Social choice according to Average without misery 


FEHJDG 


Social choice according to Least misery 


AEIBDF 


Social choice according to Most pleasure 


EFHDJA 


Social choice according to Utilitarian additive 


AIEFD 


Social choice set according to Plurality voting and Copeland rule, but in another sequence with 
the most negative items for Adam at the start. Together with EAFID and AEFID used to see if the 
order of the sequence influences the results. 



Sequences have been kept as short as possible, not to overburden the experimental subjects. Some of the 
sequences are longer (6 items) than others (5 items), because they would not distinguish between voting 
strategies otherwise. Though we used "I", "Friendl" and "Friend2" in the text of the experiment, we will use 
"John", "Adam" and "Mary" in our discussion (this makes it easier to compare between experiments). 
Note: as before we have chosen for an indirect experiment: rather than having an actual group sit down and 
measure how satisfied each individual would be with a certain sequence, subjects were asked how satisfied they 
thought all members of the group would be. Measuring satisfaction would require subjects to really experience 
the clips, and this would have required a set of clips with ratings accurately reflecting our subjects' tastes. To 
compare multiple sequences, we either would need different groups of people with the same tastes (difficult to 
find and control) or use a within subject design. However, order effects could have been large, as showing a clip 
multiple times would influence its rating. 

Satisfaction functions we will use to make predictions 

We have used six simple satisfaction functions to predict the outcomes of this experiment: 

1 . Linear Addition without Normalization: The most basic satisfaction function discussed in Section 2.4, where 
the individual's ratings of the selected items are summed. 

2. Linear Addition with Normalization: As the previous one, but now with normalization. 

3. Quadratic Addition, Pleasure only, without Normalization: Ratings are transformed as described in Section 
2.4 (10 becomes 25, 1 becomes -25 etc). The individual's positive ratings of the selected items are summed. 
Negative ratings are ignored. 

4. Quadratic Addition, Pleasure only, with Normalization. 

5. Quadratic Addition, Pleasure minus Misery, without Normalization: As Quadratic Addition, Pleasure only, 
but now the negative ratings are also incorporated in the sum. 

6. Quadratic Addition, Pleasure minus Misery, with Normalization. 

The differences in predictions between 1,3,5 on the one hand and 2,4,6 on the other hand will provide insight 
into whether subjects use Normalization. The differences in predictions between 3,4 on the one hand, and 5,6 on 
the other hand will provide insight into whether the subjects deduct misery. The differences in predictions 
between 1,2 on the one hand, and 3,4,5,6 will provide insight into whether subjects use the ratings as linear. 

We have not used the order of the sequence as input for our satisfaction functions. We did consider adding a 
satisfaction function that uses the sequence of the ratings: increasing the added satisfaction of an item if the 
previous rating was high. However, we decided to leave this to future research, as we want to resolve some of 
the other issues first, and did not know enough yet about the influence ordering could have. 



Predictions of the Satisfaction Functions 



Linear Addition with Normalization (left) and without Normalization (right) 




□ John 
■Adam 

□ Mary 



/////// 




Quadratic Addition, Pleasure Only, with Normalization (left) and without Normalization (right) 
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Quadratic Addition, Pleasure minus Misery, with Normalization (left) and without Normalization (right) 




□John 
■Adam 

□ Mary 




□ John 
■ Adam 

□ Mary 



Some of the more prominent differences: 

• Effect of Normalization. (Comparing the left hand graphs with the right hand graphs). 

FEAHD, AIEFD, AEIBDF: John beats Mary without normalization, Mary beats John with normalization. 
EFHDJB: John beats Mary without normalization, Mary and John almost equal with normalization. 
FEHJDG: Adam beats John with normalization. John and Adam quite equal without. John beats Mary 
without normalization, and they are quite equal with normalization. 

• Effect of deducting Misery. (Comparing the middle two graphs with the bottom two graphs). 
EFHDJA: Adam beats Mary without deduction of misery. With deduction, Mary beats Adam (or is almost 
equal to Adam, without normalization). 

• Effect of Quadratic versus Linear. (Comparing the top two graphs with the bottom four graphs). 
FEHJDI: In Linear Addition, John and Mary are quite equal. In the others, John clearly beats Mary. 
FEHJDG: In Quadratic, Adam clearly beats Mary. In Linear, Adam and Mary are quite equal. 



EFHDJA: In Linear, Mary beats John (or is equal to John without normalization). In Quadratic, John beats 
Mary. 



Research questions 

1. Is one of our satisfaction functions a good predictor of subject behavior? The satisfaction functions above 
predict how each individual's satisfaction compares to that of the other two individuals for a particular 
sequence, as well as how it compares to their own satisfaction for other sequences. For instance, according 
to the Quadratic-Addition-Pleasure-Only satisfaction functions, John and Mary would be more satisfied with 
AIEFD than Adam, and both would prefer AIEFD to FEHJDL The predictions of the various satisfaction 
functions differ, and in the experiment we can compare these predictions with the satisfactions as indicated 
by our experimental subjects. A main question was whether the predictions of one our satisfaction functions 
would closely match those of our subjects. We also wanted answers to the following related questions, to 
better understand what makes a good satisfaction function: 

• Do subjects use normalization! 

• Do subjects deduct misery! 

• Do subjects use linear rating scales! 

We can use the differences in predictions as discussed above to answer these three questions. 

If one of the satisfaction functions is a very good predictor, then it might be possible to also determine 

thresholds. Let s be the predicted satisfaction of an individual with a sequence. We would like to determine 

thresholds tl, t2, t3, t4 and t5, such that: 

s < tl : individual is very dissatisfied with the sequence (score of 1 on the scale of the experiment) 
tl < s < t2 : individual is dissatisfied with the sequence (score of 2 or 3 on the scale) 
t3 < s < t4: individual is satisfied with the sequence (score of 5 or 6 on the scale) 
t4 < s : individual is very satisfied with the sequence (score of 7 on the scale) 

If one of the Normalized satisfaction functions were a good predictor, then all thresholds would have to be 

between 0 and 1 . 

2. Does the order of the sequence influence subjects' satisfaction ratings? What aspects of order do subjects 
mention? We have given the subjects sequences of selected items, rather than sets. This allows the subjects 
to use this information if they want to. We hope that the subjects' explanations will provide us with more 
insight into how order influences satisfaction. To study this further, we have included three experimental 
conditions (EAFID, AEFID, AIEFD) that are exactly the same, with the same set of items being selected, 
except that they are presented in a different order. Note that in our experiments we have deliberately 
abstracted items (A to J) instead of telling the subjects about the item content. This ensures that subjects use 
the ratings provided rather than their own personal opinions. This has as side effect that the subjects cannot 
use content information (the emotional tone or how the content is related to the content of other items). So, 
our experiment will be restricted to the impact of the ratings profile of a sequence on satisfaction (for 
instance, exploring the difference between "7 1 93 9" and "1 3 79 9"), and not the emotional or content 
profile. It might be possible to change this in a future experiment, but it is a tricky issue to handle: for 
instance, telling subjects an item is 'sad' might influence their opinion of that item in isolation, not just as 
part of a sequence. 

3. Do subjects use social aspects to determine individual satisfaction? We have not used social aspects 
(solidarity or jealousy) in our satisfaction functions. Without having resolved the other issues first, it would 
have led to an explosion in possible satisfaction functions. Even more so, as the selfishness/sociality of our 
individual subjects would have an impact, and different 'average sociality' weightings would have been 
needed for the predictions. However, we hoped that subjects' explanations would indicate whether they had 
taken social aspects into account. 

4. Is there one strategy (or multiple ones) that is clearly better than the others, in terms of keeping all members 
of the group happy? As subjects would indicate the satisfaction of all members of the group, we could 
investigate whether there is a sequence (i.e. the result of a particular selection strategy) with positive 
satisfaction (i.e. score 5 or above) on average —or better even for all subjects— for all members of the group. 

Subjects 

Twenty-two subjects participated in the experiment. All were academic staff of the University of Brighton. 
Subjects were assigned to experimental condition at random. To control for order effects, each sequence 
appeared similarly often as first, second and third paper. Permutated sequences AIEFD, AEFID, and EAFID 
were not given to the same subjects, so, there was a complete between-subjects design for those. Each sequence 
was studied by at least seven subjects (some by eight). 



4.2 Results and discussion 

The figure below shows the average satisfaction scores per condition. 




Do subjects use normalization? 

Comparing the results with the prominent differences between the predictions of the satisfaction functions (see 
Section 4.1), we find: 

• FEAHD: There is a trend that Mary beats John, as predicted by normalization. This is not statistically 
significant, but "without normalization" predicted John to beat Mary. 

• AIEFD, AEIBDF: John and Mary are quite equal, which is more in tune with normalization. 

• EFHDJB: Mary and John almost equal, as predicted by normalization. 

• FEHJDG: There is a trend that Adam beats John, as predicted by normalization, but this is not statistically 
significant. However, John clearly beats Mary, as predicted by "without normalization" (this is statistically 
significant, p<0.01). 

So, overall, there is some evidence that normalization has taken place, but with a contradictory result for 
FEHJDG (which is a strange case, as will be discussed below). 



Do subjects deduct misery? 

Comparing the results with the predictions of the satisfaction functions (see Section 4.1), we find: 

• EFHDJA: There is a trend that Mary beats Adam as predicted by deducting misery. This is not statistically 

significant, but without deducting misery it was predicted that Adam beats Mary. 
So, there is some evidence that misery is taken into account (deducted). This is in line with the results of 
Experiment 1 . 



Do subjects use linear rating scales? 

Comparing the results with the predictions of the satisfaction functions (see Section 4.1), we find: 

• FEHJDI: John beats Mary, as predicted by Quadratic. This is statistically significant (p<0.01). 

• FEHJDG: Adam clearly beats Mary, as predicted by Quadratic. This is statistically significant (pO.OOl) 
So, there is clear evidence that Quadratic is a better measure than Linear. 



Does the order of the sequence influence satisfaction? 

Order is mentioned by six subjects: 

• One subject (S22) mentioned that the first clip mainly influenced his ratings. Another subject (S3) explained 
a low rating for Adam in AIEFD: "had to endure his two least favorite clips. Then he gave up and went 
away." 

• One subject (S20) considered how the individuals' experience changed over time. For FEHJDH he mentions 
the problem of John feeling a decline of "quality" from the second clip onwards. He also mentioned that 
John would be quite satisfied with EFHDJA "ending in a high" (in addition to seeing good clips). In support 
of this, another subject explained a low satisfaction of Adam with EFHDJA by "never finish on a bad one" 



(S7). A third subject (SI 8) shared this interest in the end of the sequence: she mentioned that Adam and 
Mary would be more satisfied than John with FEAHD (contrary to general belief) because they get more 
favorite clips towards the end. She also gave John a low rating (3) for EFHDJA because she assumed it to be 
boring to see too many favorites in a row! 

• One subject (SI 3) explicitly indicated not having taken order into account. 

No statistically significant differences were found between the results of EAFID, AEFID, and AIEFD. This is 
not surprising given the small number of subjects who mentioned order. Most subjects treated the sequences as 
sets. The ordering issue will need to be studied further in future experiments, particularly investigating the 
impact of the start of the sequence, the end, and the increase or decrease of pleasure over time. 

Do subjects use social aspects to determine individual satisfaction? 

Most subjects did not explicitly do so. 

• Only three subjects (S 1 5, S 1 2, and S 1 1 ) mentioned taking the mood of the others into account when 
determining the satisfaction of an individual. Another subject (SI 6) mentioned "If satisfaction depends on 
others being satisfied I could not assess this, because there was too much to hold in short-term memory. I 
had no overall model of the situation, and merely did the three evaluations independently." 

• One of these subjects (SI 1) mentioned that it would be important to know more about the context: "if 
watching at your own house, your visitors' satisfaction becomes more important". 

Is one of the satisfaction functions a good predictor? 

Comparing the average profile above with the profile of the normalized "Quadratic Addition, Pleasure minus 
Misery" satisfaction function, it can be concluded that they are quite similar. There are, however, still a number 
of noticeable differences: 

• Adam's satisfaction for AIEFD (and its permutations) is low, but not as low as predicted. Similarly, Adam's 
satisfaction for AEIBDF is higher than expected. Perhaps the numbers associated with low ratings (like -25 
for a rating of 1) should be less negative than we have them. So, deducting misery, but less severely. 

• The difference between John's and Mary's satisfaction for FEHJDG is larger than expected. Comparing 
Mary's satisfaction for FEHJDG with that for EFHDJB shows a significant difference (p<0.05). This is 
strange as the only difference between both sequences (except order) is the G in one sequence and the B in 
the other, both of which have the same rating for Mary. Perhaps subjects did take a social aspect into 
account: preferring B (which at least gives pleasure to Adam) to G (which gives pleasure to nobody), and 
feeling more upset about missing favorite A because G pleases nobody. Many subjects mention missing A 
as the reason for giving a low mark. 

• John's satisfaction for EFHDJA is lower than expected (compared to Adam's and Mary's satisfaction). This 
might be partly caused by the somewhat odd behavior of subject SI 8 (See order section above). Another 
subject mentioned giving John a low rating (3) because of missing out on one of his favorites (I). 

• The last two points (and the frequency with which subjects used the argument of missing the favorite for 
giving a lower mark) indicates that a higher weight has to be given to favorites, such that satisfaction goes 
down when all favorites are missed. 

These differences have made it impossible to determine thresholds. 

Is there a strategy that keeps everyone happy? 

On average, John, Adam and Mary were all reported to be not dissatisfied with FEAHD (Borda Count), FEHJDI 
(Multiplicative Utilitarian), EFHDJB (Average without Misery), AEIBDF (Most pleasure) and EFHDJA 
(Additive Utilitarian). Looking at individual subjects' responses, FEHJDI (Multiplicative Utilitarian) is the only 
sequence that has ratings of at least 4 for all subjects for all individuals. The average ratings for this sequence are 
even above 5 for John, Adam, and Mary, showing a certain degree of satisfaction for all of them. So, 
Multiplicative Utilitarian seems the most promising strategy, but the others are not bad either. Of course, we 
need to investigate this issue further, using different individual ratings and different lengths of the sequence. 
However, on the basis of this experiment, we can reduce the number of strategies to be investigated (discarding 
Copeland rule, Plurality voting, Least misery). 

Other issues 

• One subject mentioned that the length of the clips would be important. (S19) This is a valid point, as 
viewing something you hate for five minutes or an hour would indeed make a big difference. 



• One subject mentioned that other factors would influence satisfaction, like the discussion on the basis of the 
clips (SI 8). 

• Almost all subjects talked about including (or not having included) the favorite clip(s). This seemed to be a 
more important issue than expected. 



5 Algorithms for presenting a sequence: the issues of order and ratings 

Until now we have focused on how a set of items suitable for a group can be selected based on the individuals' 
ratings. The television will also need to decide in which order to show the items. In this section, we will sketch 
three algorithms that take ordering issues into account, and we will empirically explore the assumptions 
underlying these algorithms. 



5.1 Algorithms 

Algorithm 1: Using the group list ranking 

As the selection strategies described produce a ranked group list, the simplest algorithm is to show the items in 
the same order as they appear in the list. We have applied this method when producing the sequences for 
Experiment 2. The algorithm is depicted in Figure 1 . Note that this algorithm might need to be slightly modified 
if the time durations of items vary. Television programmes (such as the news) tend to have a fixed length. Items, 
on the other hand, could have varying lengths, with one item being longer than another. It is possible that at the 
end of the sequence an item lower in the group list ranking needs to be selected to fill the available time, as the 
item which turn it was might be too long. 



Determine 
Group List 



Show first 
item of list 



Remove item 
from list 




Figure 1. Algorithm for deciding sequence using group list ranking. 



Algorithm 2: Changing order 

Using the group list ranking to determine the order will produce a sequence with items liked by the group at the 
start and a decline in quality afterwards. However, this might not be the best order for keeping everybody 
optimally satisfied. As indicated in Section 2.2 we believe that a good sequence should keep each individual's 
satisfaction above a threshold at each moment during the broadcast. Our example in that section already showed 
that the order of the items impacts whether or not a sequence meets this criterion. Comments made by some 
subjects in the previous experiment show that there might be additional aspects that decide how good a sequence 
is, like having a strong ending. As discussed in Section 2.5, it is also likely that the mood induced by watching an 
item and the topic of an item influences what is the best item to show next. So, instead of showing items in the 
order of the group list; a more advanced algorithm could merely use the group list to decide which set of items to 
show and then order this set taking certain constraints into account. For instance, it could order the set such that 
the sequence 

• Keeps individuals sufficiently satisfied throughout the broadcast: the predicted satisfaction of each 
individual at each point in the sequence is above a certain threshold 

• Has a strong ending: The predicted satisfaction of each individual with the last M items is above a certain 
threshold 

• Exhibits consistency in mood: The predicted mood induced by each two adjacent items in not "too wide 
apart" on a mood scale. For instance, it might be fine to succeed a "Very Happy" item with a "Happy" item, 
but not with a "Very Sad" item. 

• Has a good narrative flow: Topically related items are as close as possible in the sequence. 

Note that these constraints are both speculative and vague. Though they might seem reasonable, we have not 
proven yet that they are, and we have not specified them in detail (for instance, how far apart are two adjacent 
items allowed to be on the mood scale, what is the value of M, etc.). Also, the constraints could conflict: two 



items can be highly topically related but induce widely different moods. In the experiment below we will explore 
our assumptions that mood and topical relatedness should have an impact on the order. 
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Figure 2. Algorithm for deciding sequence using ordering constraints. 



Algorithm 3: Changing ratings 

There seems to be one major flaw in the reasoning above: until now we have tacitly assumed that we can first 
select a set of items and then order them, ignoring that ratings might change as an effect of viewing an item. 
Having shown an item to the group, issues like mood consistency and topical relatedness might well lead to an 
item outside the selected set being more suitable to show next than the items in our selected set. We believe that 
the individuals' ratings should be recalculated taking into account the items they have seen so far. This leads to 
the algorithm depicted in Figure 3. Different rules could be used for the recalculation of ratings. For instance: 

• If an item is topically related to the item shown, then increase its rating (by an amount proportionate to the 
relatedness). 

• If an item has the same mood as the item shown, then increase it's rating, and decrease the rating if the 
moods conflict (by an amount proportionate with the intensity of the mood). A question is whether changes 
in ratings need to apply only for the duration of the next selection or for longer. For instance, suppose the 
TV shows a "very sad" item. This might lead to a reduction in ratings for "happy" items. If the TV were to 
show a "neutral" item next, should the ratings for the "happy" items be restored to their previous value, or 
remain reduced? 

• If the (predicted) viewer's satisfaction is high after watching the items so far, then increase ratings, and if 
the (predicted) satisfaction is low then decrease ratings. To give an example of how this might work: 
Assume Adam is shown a number of items he likes. This gives him high satisfaction. Because of this, his 
ratings for other items (like the ones he normally hates) will increase. This makes it more likely these items 
will be selected (for instance, by passing the threshold in an Average Without Misery strategy). Assume as a 
consequence an item he normally hates is shown, like item A. This reduces his satisfaction and therefore his 
ratings for other items, making it more likely he will see something he enjoys soon. 
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Figure 3. Algorithm for deciding sequence using rating modification. 



Note that this algorithm allows items to be shown multiple times. This can be counteracted by sharply decreasing 
the rating of items that have been shown, or by adding a constraint to the "Show first item of the list" that it has 
to be the first item not already shown. Whilst showing a news item multiple times needs to be avoided, this 
might not be equally true for all types of programmes and viewers. Toddlers, for instance, very happily watch 
something they like multiple times. Similarly, a music clip might warrant watching multiple times for a fan. It 
seems therefore best to leave this issue to be dealt with by the sub-algorithm that determines each individual's 
ratings. 

To investigate whether this algorithm is indeed better than the other two algorithms discussed, our next step is to 
explore if people do indeed adjust their ratings. More particularly, we would also like to gain insight into the 
effects of mood, satisfaction, and topical relatedness on the ratings of items after having viewed another item. 



5.2 Experimental design 



Method 

Subjects were told that they were going to watch the evening news, and that the television would select a 
sequence of news items for them. They were asked to rate seven news items on how much they would want to 
watch them and how they might expect the news item to make them feel. After they had rated all seven items, 
subjects were told what the first item on the news was, an item that they had not yet seen. They were asked to 
rate this first item on how much they would want to watch it and how they might expect the item to make them 
feel. Next, they were asked to rate the other seven news items again, given that they had just watched that first 
item. There were two experimental conditions. In Condition A, the first news item was "Brighton University 
Watts Building on Fire: hundreds feared dead". In Condition B, it was "England football team has to play 
Bulgaria in the next round". (See Appendix C for the exact wording.) The table below shows the news items 
used, and the reason for using them. 



ll V TV 3 llvlll 


Reason 


"[Insert name of your favorite sport's club] wins 
important game" 


Expect that most subjects would want to see this item 
and that it would make them feel very happy. 


"Fleet of limos for Jennifer Lopez 100-metre trip" 5 


This item and the next two were real news items that 
were chosen to he not tonicallv related to the "Eneland 
football team has to play Bulgaria in the next round" 
and "Brighton University Watts Building on Fire: 
hundreds feared dead" items. 


"Heart disease could be halved" 5 


Expect that it would make most subjects happy. 


"Is there room for God in Europe?" 5 


Expect that subjects would differ in opinion. 


"Earthquake hits Bulgaria" 


Expect that most subjects would want to see this item 
and that it would make them feel sad. Also picked 
"Bulgaria" to make this item weakly topically related 
to the "England football team has to play Bulgaria in 
next match" item. 


"UK fire strike continues" 


Expect that most subjects would not want to see this 
item at the start. The UK fire strike had already been in 
the news for months. Item is weakly topically related 
to "Brighton University Watts Building on Fire: 
hundreds feared dead" item. 


"Main three Bulgarian players injured after Bulgaria- 
Spain football match" 


Expect that most subjects would not want to see this 
item at the start. Item is strongly topically related to 
the "England football team has to play Bulgaria in next 
match" item. 


"Brighton University Watts Building on Fire: 
hundreds feared dead" (only in Condition A) 


Most of the subjects' lectures take place in Watts 
Building. We therefore expected them to be both very 
interested in and very sad about this item. Constructed 
to investigate the effect of mood. 


"England football team has to play Bulgaria in the 
next round" (only in Condition B) 


Topically related to two other items, with a different 
degree of relatedness. Constructed to investigate the 
effect of topical relatedness. 



Research questions 

We wanted answers to the following research questions: 

• Do people adjust ratings? Does having watched the first news item influence the ratings for the other news 
items? If this is the case, then a new group list will have to be determined after each presented item, as in 
Algorithm 3 of the previous section. If the ratings stay the same, then we could apply an ordering algorithm 
to the group list, as in Algorithm 2 of the previous section. Our hypothesis was that in both conditions the 
ratings would change. 

• Does mood influence the way ratings are adjusted? Condition A was constructed to test this. We expected 
all subjects to feel very sad after viewing the "Brighton University Watts Building on Fire: hundreds feared 



5 These news headlines were taken from the Yahoo news site http://uk.news.yahoo.com/ on 28-2-2003. 



dead" item. Our hypothesis was that this would influence their ratings, particularly of items of a conflicting 
("happy") mood, such as the "[Insert name of your favorite sport's club] wins important game" item. 

• Does topical relatedness influence the way ratings are adjusted? Does having watched the first news item 
influence the ratings of topically related items to a higher extent than the ratings of topically unrelated 
items? Condition B was constructed to test this. Our hypothesis was that after watching the "England 
football team has to play Bulgaria in the next round" item, subjects would change their ratings for the "Main 
three Bulgarian players injured after Bulgaria-Spain football match" item, and, to a lesser extent, for the 
"Earthquake hits Bulgaria" item. 

• Does the subject 's satisfaction influence the way ratings are adjusted? Does a subject with a high rating for 
the first item increase the ratings of the other items? Does a subject with a low rating for the first item 
decrease the ratings of the other items? We can only test this on items that are topically unrelated and in the 
same mood (or these aspects might cause a change). Our hypothesis was that the rating for the "Earthquake 
hits Bulgaria" item would increase after having seen the "Brighton University Watts Building on Fire: 
hundreds feared dead" item, as both are sad items, topically unrelated, and the "on fire" item is expected to 
have a high interest rating. 

• Is there an interaction between these factors? Is there an interaction between the subject's satisfaction of an 
item and the effect of topical relatedness on ratings for other items? Our hypothesis is that subjects who 
express an interest in the "England football team has to play Bulgaria in the next round" item are more 
likely to increase the rating of the topically related "Main three Bulgarian players injured after Bulgaria- 
Spain football match" item than subjects who were not interested. 

Subjects 

Thirty-four subjects participated in the experiment. All were final-year undergraduate students of the IT faculty 
attending a lecture of the Adaptive Interactive Systems module 6 . The students were studying various courses 
(B.A. Computer and Information Systems, B.Sc. Computer Studies, B.Sc. Computer Science, and B.Sc. 
Software Engineering). The experiment took place in a lecture room. Subjects were randomly assigned to an 
experimental condition. Students participated in the experiment voluntarily (in addition to the numbers 
mentioned above, two students chose not to participate). The spread over courses was similar for both 
conditions. 



5.3 Experimental results and discussion 

Influence of viewing an item on ratings for other items 

The results for Condition A ("Brighton University Watts Building on Fire: hundreds feared dead") clearly 
confirm our hypothesis that there can be a large influence of viewing an item on the ratings of other items. 
Figure 4 shows a dramatic decrease in ratings after seeing this item. All subjects modified at least one rating, and 
the difference in ratings was statistically significant. We can therefore conclude that Algorithm 3 is the best 
candidate. 

Influence of mood 

The results for Condition A ("Brighton University Watts Building on Fire: hundreds feared dead") were hoped to 
shed some light on the influence of mood. We clearly succeeded in producing an item that made all subjects 
expect to feel sad. AH subjects chose "very sad", with the exception of two who chose the category just above 
that. However, we probably succeeded too well: subjects were expecting to be sad to the extent that most 
decreased their ratings across the board. Some subjects expressed this by making comments like "I would not be 
interested to see anything anymore, when worried about friends dying". This does confirm our hypothesis that 
mood can influence ratings, but not our more particular hypothesis about the way it would affect them. We did 
not find that the ratings of happy items decreased more than those of sad items. On the contrary, the Heart 
Disease item (judged by subjects as making them happy) showed a smaller decrease than the sad Earthquake 
item. So, contrary to our hypothesis and the literature described in Section 2, it might be that subjects actually 
prefer a happy item to distract them from the sad news. Subjects' comments also indicate that there is more to 
items than mood and topic: namely, an importance dimension. Subjects commented that after seeing the 



6 Note: this experiment took place almost a year after Experiment 1, so none of these subjects had participated 
before. 



Brighton University On Fire item, they were no longer in the mood for irrelevant items, such as the Wins Game 
and Lopez items. 




News item 


Mean (StD 
Before 


ev) 
After 


Wins Game 


4.71 (1.99) 


3.65 (2.18) 


Lopez 


3.18(1.78) 


2.47(1.74) 


Heart Disease 


5.53 (1.07) 


4.76(1.39) 


God 


3.94(1.89) 


3.06(1.78) 


Earthquake 


5.47(1.28) 


4.00(1.41) 


Fire Strike 


4.71 (1.26) 


3.65(1.37) 


Player Injury 


2.41 (1.42) 


2.00(1.37) 



News Item 



Figure 4. Box plot and summary table for Condition A showing the mean, standard deviation, median, and 
quartiles, and outliers (circles) of the subjects' ratings for the various news items before and after seeing the 
"Brighton University Watts Building on Fire: hundreds feared dead" item. 



We also did a small pilot test using an -as we thought- positive mood inducing item "War avoided by 
negotiation". This proved problematic: our pilot subjects varied in opinion on how happy this item would make 
them feel. Their comments show that this was due to the subjects' opinion on the desirability of avoiding a war 
in Iraq. Note that we never specified which war was supposedly avoided. This shows the difficulty in 
construction items with a certain fixed mood or interest. 



Influence of satisfaction 

We could not draw any conclusions about the influence of satisfaction from the "Earthquake in Bulgaria" item in 
Condition A, because the "University on Fire" item had had the unexpected effect of changing all ratings. Also, 
no effect was found in Condition B. Therefore, we have not found any proof to support the recalculation of 
ratings dependent on the rating of the previous item. However, we need still to investigate whether satisfaction 
would have an influence if there were no other influencing factors (like mood and topical relatedness). It might 
also be that the influence of satisfaction is not a conscious influence, one that people are aware of, and that a 
more realistic experiment in which subjects were really viewing video clips is needed. 

Influence of topical relatedness 

The results for Condition B ("England football team has to play Bulgaria in the next round") confirm our 
hypothesis that viewing an item can influence the ratings of topically related items. Figure 5 shows that subjects 
increased the ratings of the topically related items, while the ratings of the topically unrelated items remained 
similar (actually, the means even show a small decline in ratings for these items). The strongly topically related 
"Main three Bulgarian players injured after Bulgaria-Spain football match" also showed a larger change in 
ratings than the weakly related "Earthquake hits Bulgaria". 

Interaction between satisfaction and the effect of topical relatedness 

We expected the effect of topical relatedness to depend on the interest the subject had for the item they viewed. 
For instance, if a subject were completely uninterested in the English football team, then we would not expect an 
effect on their ratings for the related "Main three Bulgarian players injured after Bulgaria-Spain football match". 
On the contrary, if a subject were very interested in the English football team, then we would expect their ratings 
for the related item to go up. Figure 6 shows the effect of watching the first (England-Bulgaria) item on the 
topically related items (Bulgarian Player Injury, and Earth Quake in Bulgaria) as a function of the subjects' 



ratings for the first item. The figure to a certain extent supports our hypothesis, with the increases in ratings 
mainly for subjects who expressed some interest (rating 5 and above), and the decreases in ratings mainly for 
subjects who were not that interested (ratings 4 and below). 




News item 


Mean (St.D 
Before 


ev) 
After 


Wins Game 


5.06(1.92) 


5.18(1.70) 


Lopez 


2.47(1.51) 


2.29(1.40) 


Heart Disease 


5.76 (.97) 


5.59(1.06) 


God 


3.47(1.81) 


3.47(1.66) 


Earthquake 


4.76(1.52) 


4.88(1.36) 


Fire Strike 


4.47(1.59) 


4.41 (1.66) 


Player Injury 


3.76(1.44) 


4.18(1.63) 



News Item 

Figure S.Box plot and summary table for Condition B showing the mean, standard deviation, median, quartiles, 
and outliers (circles) of the subjects' ratings for the various news items before and after seeing the "England 
football team has to play Bulgaria in the next round" item. 




Figure 6. Rating differences for "Main three Bulgarian players injured after Bulgaria-Spain football match" 
(left) and "Earthquake hits Bulgaria" (right) after seeing the "England football team has to play Bulgaria in the 
next round" news item (y-axis), plotted against the rating of the subject for the latter (x-axis). Size of the bubbles 
indicates number of subjects. 



6 Conclusions 

Group modeling is an interesting research area with a wide possibility of applications, both in interactive 
television and beyond. In this paper, we have defined the problems associated with adaptation to groups, 
described our initial research in this area, discovered some answers and many more questions that need 
answering. 



6.1 Conclusions from the study 



The main results from this study are: 

• People use some of the choice strategies described in Section 2, such as the Average Strategy (a.k.a. 
Additive Utilitarian), the Average Without Misery Strategy and the Least Misery Strategy, and care about 
fairness and avoiding individual misery. (Experiment 1) 

• People use normalization: their satisfaction does not only depend on the selected items, but also on the not 
selected ones. (Experiment 2) 

• People deduct misery: their satisfaction is negatively affected by disagreeable experiences, more than 
because of losing out on possible pleasure. (Experiment 2) 

• People use the ratings in a non-linear way: i.e. the difference in ratings between say 9 and 10 is regarded as 
larger than that between 7 and 8. (Experiment 2) 

• The "Normalized Quadratic Addition, Pleasure minus Misery" satisfaction functions produced reasonable, 
though not completely correct predictions. (Experiment 2) 

• The sequences produced by five choice strategies (Borda Count, Multiplicative Utilitarian, Average without 
Misery, Most pleasure, and Additive Utilitarian) gave, in our example, on avarage satisfaction to all 
individuals in the group. (Experiment 2) 

• Multiplicative Utilitarian seems the best strategies as it's sequence produced satisfaction for all individuals 
in the group according to all subjects. (Experiment 2) 

• People's opinion about items can change dramatically as a result of watching another item. Hence, ratings 
need to be recalculated after showing each item, and a new Group list needs to be determined before 
selecting the next item. (Experiment 3) 

• People's opinion about items can change as a result of the mood induced by watching an item. Watching a 
very sad item can decrease the ratings for other items. Contrary to expectation, it does not seem to decrease 
the ratings of conflicting mood items more than those of similar mood. (Experiment 3) 

• People's opinion about items can change as a result of their topical relatedness to a shown item. There is an 
interaction with Satisfaction: if the person is interested in the shown item, than the ratings of topically 
related items are more likely to increase. (Experiment 3) 

6.2 Limitations of the study 
Indirectness of the experiments 

As discussed in Sections 3 and 4, we have deliberately chosen to make the experiments indirect. In Experiment 
1, rather than having an actual group sit down to decide what to watch, subjects were asked what they thought 
people should watch. In Experiment 2, rather than having an actual group sit down and measure how satisfied 
each individual would be with a certain sequence, subjects were asked how satisfied they thought all members of 
the group would be. In Experiment 3, real news headings were used for items, rather than the abstract items used 
in the earlier experiments. But again, we did not really show the items to the subjects, but asked them how much 
they wanted to see them and how they expected viewing them would make them feel. We explained the reasons 
for this indirect approach in Sections 3 and 4, and they mainly had to do with controlling the experiment. 
However, as always, the more an experiment is controlled, the less it resembles the real world. 

Subjects used in the experiments 

The sets of subjects were quite homogeneous, particularly the level of education: either having or studying for a 
degree, mostly in computing. Subjects were also from a relatively narrow age range (though not as narrow as the 
word "student" suggests, as we have a large proportion of mature students), and the majority of subjects in 
Experiments 1 and 3 were male. This raises some doubts about the general isability of the results, as the 
demographic of a television audience is a lot more heterogeneous. However, as the indirectness of our 
experiments did require a certain level of education, it would have been impossible to use a cross section of the 
population. This would also have required a rather large group of subjects. Nevertheless, the general isability of 
the results remains to be proven. 

Example used in the experiments 

In both Experiments 1 and 2, we have used the example of a group of three people, with particular ratings for 
these three people. It still needs to be proven that our result about the suitability of the strategies is general isable 
to larger groups, and different rating distributions. 



Assumptions made 

As discussed in Section 2, we have made a number of assumptions. In a sense, all of them are limitations of this 
study. The most important limitation seems our assumption that ratings are accurate: recommender systems need 
to be able to deal with uncertainty. This might mean that rather than having an accurate rating for an individual 
for an item, we might have a probability distribution that indicates the likelihood of certain ratings for that item. 
Or it might mean that we have an estimated rating with an indication of how certain the system is about its 
estimation. Taking uncertainty into account would have made our experiments far more complicated. We had, 
therefore, decided to separate concerns, and start with the assumption that ratings were accurate. We believe that 
most results -in the sense of what is important to people, like avoiding misery- would still hold when dealing 
with uncertainty, but that an additional set of rules would apply in that case. 



6.3 Areas for further work 

Our research has only just started, and raises many questions that warrant further research. For example: 

• The Multiplicative Utilitarian strategy seems a good strategy to use, but more experiments are needed to 
confirm this. These experiments would need to deal with the limitations mentioned above. 

• One way to reduce the effort involved in empirical evaluations would be to have a good Satisfaction 
Function to predict experimental results. The "Normalized Quadratic Addition, Pleasure minus Misery" 
satisfaction function is a promising start, but needs further improvement to become more accurate. 

• Finding a highly accurate Satisfaction Function would also allow us to mathematically determine the 
optimal strategy. 

• The ordering of the sequence requires more investigation. This is not only a group adaption issue, but 
applies also when dealing with only one viewer. We need to determine what exactly the effect of mood is, 
how to correctly predict the size of the effect of topical relatedness on ratings, and how to deal with the 
importance dimension. 

• Invisible members can be added to a group (to represent teachers, or parents) to ensure that a viewer's 
(student or child) overall viewing experience is more appropriate. It should be investigated how this can be 
done and can be made both acceptable and beneficial for the viewer. Similarly, television critics could be 
added as members of a group. Their ratings would accurately reflect their opinions. 

• An individual's satisfaction might be influenced by adequate user interface design. For instance, when 
showing an item, it could be indicated to the viewers what the next item(s) will be (for instance, using a 
subtitle). This tells viewers who do not like the current item that the next one will be to their taste. This 
might avoid dissatisfaction, boredom, and walking away from the television. 
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Appendix A 



Scenario 1 

John, Mary, and Adam are going to watch video clips together. We know how interested they are in the topics of 
the clips. Each clip is rated from 1 - really hate this topic - to 10 - really like this topic. 



They only have time to watch one clip. Which clip 
should they watch? Why? 

They only have time to watch two clips. Which clips 
should they watch? Why? 

They only have time to watch three clips. Which clips 
should they watch? Why? 

They only have time to watch four clips. Which clips 
should they watch? Why? 

They only have time to watch five clips. Which clips 
should they watch? Why? 

They only have time to watch six clips. Which clips 
should they watch? Why? 

They only have time to watch seven clips. Which clips 
should they watch? Why? 



Scenario 2 

John (29), Mary (32), and their grandfather Adam (81) are going to watch video clips together. We know how 
interested they are in the topics of the clips. Each clip is rated from 1 - really hate this topic - to 10 - really like 
this topic. 



They only have time to watch one clip. Which clip 
should they watch? Why? 

They only have time to watch two clips. Which clips 
should they watch? Why? 

They only have time to watch three clips. Which clips 
should they watch? Why? 

They only have time to watch four clips. Which clips 
should they watch? Why? 

They only have time to watch five clips. Which clips 
should they watch? Why? 

They only have time to watch six clips. Which clips 
should they watch? Why? 

They only have time to watch seven clips. Which clips 
should they watch? Why? 



Clip 


John 


Adam 


Mary 


A 


10 


1 


10 


B 


4 


9 


5 


C 


3 


8 


2 


D 


6 


9 


7 


E 


10 


7 


9 


F 


9 


9 


8 


G 


6 


6 


5 


H 


8 


9 


6 


I 


10 


3 


7 


J 


8 


8 


6 



Clip 


John 


Adam 


Mary 


A 


10 


1 


10 


B 


4 


9 


5 


C 


3 


8 


2 


D 


6 


9 


7 


E 


10 


7 


9 


F 


9 


9 


8 


G 


6 


6 


5 


H 


8 


9 


6 


1 


10 


3 


7 


J 


8 


8 


6 



Appendix B 



Age: Gender: M / F 

You and two friends (Friend 1 and Friend2) are going to watch video clips together. The Television knows 
how interested you are in the topics of the clips. Each clip is rated from 1 -really hate this topic- to 10 - 
really like this topic. 



Clip 


You 


Friendl Friend2 






i n 1 




B 


4 


9 


5 




iD m 1 




Li , -a 1 


D 


6 


9 


7 


mmm 








F 


9 


9 


8 








^BTT ilH 


H 


8 


9 


6 






IS 1 


D 1 


J 


8 


8 


6 



It decides to show you the following sequence of clips: E A F I D 

Very dissatisfied Neutral Very satisfied 

How satisfied would you be? 1 2 3 4 5 6 7 

Why? 



How satisfied do you believe Friendl would be? 1 2 3 4 5 6 7 

Why? 



How satisfied do you believe Friend2 would be? 1 2 3 4 5 6 7 

Why? 



Appendix C 



Age: Gender: M / F 

You are going to watch the evening news. A number of things have happened today, and the news 
programme has to make a selection about what to show you. Decide for the following new items how 
interested you would be to see them, and how you think they would make you feel. Each news item is 
described by its headline, more detail would be given in the news. Assume all news items to be true. 



"[Insert name of your favorite sport's club] wins important game" 

Really Really 
Hate to Neutral Want to 

How much would you want to watch this news 1 2 3 4 5 6 7 

item? 

Very Very 
Sad Neutral Happy 

How might you expect this news item to make 1 2 3 4 5 6 7 

you feel? 

"Fleet of limos for Jennifer Lopez 100-metre trip" 

Really Really 
Hate to Neutral Want to 

How much would you want to watch this news 1 2 3 4 5 6 7 

item? 

Very Very 
Sad Neutral Happy 

How might you expect this news item to make 1 2 3 4 5 6 7 
you feel? 

"Heart disease could be halved" 

Really Really 
Hate to Neutral Want to 

How much would you want to watch this news 1 2 3 4 5 6 7 

item? 

Very Very 
Sad Neutral Happy 

How might you expect this news item to make 1 2 3 4 5 6 7 
you feel? 

"Is there room for God in Europe?" 

Really Really 
Hate to Neutral Want to 

How much would you want to watch this news 1 2 3 4 5 6 7 

item? 

Very Very 
Sad Neutral Happy 

How might you expect this news item to make 1 2 3 4 5 6 7 
you feel? 



"Earthquake hits Bulgaria" 

Really 
Hate to 

How much would you want to watch this news 1 2 
item? 

Very 
Sad 

How might you expect this news item to make 1 2 
you feel? 



Neutral 
4 5 



Neutral 
4 5 



Really 
Want to 
6 7 

Very 
Happy 
6 7 



"UK fire fighter strike continues" 

Really 
Hate to 

How much would you want to watch this news 1 2 
item? 

Very 
Sad 

How might you expect this news item to make 1 2 
you feel? 



Neutral 
4 5 



Neutral 
4 5 



Really 
Want to 
6 7 

Very 
Happy 
6 7 



"Main three Bulgarian players injured after Bulgaria-Spain football match" 

Really Really 
Hate to Neutral Want to 

How much would you want to watch this news 1 2 3 4 5 6 7 

item? 

Very Very 
Sad Neutral Happy 

How might you expect this news item to make 1 2 3 4 5 6 7 

you feel? 



The first item on the news is "Brighton University Watts Building on Fire: hundreds feared dead" 

Really Really 
Hate to Neutral Want to 

How much would you want to watch this news 1 2 3 4 5 6 7 

item? 

Very Very 
Sad Neutral Happy 

How might you expect this news item to make 1 2 3 4 5 6 7 
you feel? 



Given that you have just seen this item, how much would you now want to watch these items? 



Really 
Hate to 



"[Insert name of your favorite sport's 
club] wins important game" 

"Fleet of limos for Jennifer Lopez 
100-metre trip" 

"Heart disease could be halved" 

"Is there room for God in Europe?" 

"Earthquake hits Bulgaria" 

"UK fire fighter strike continues" 

"Main three Bulgarian players injured 
after Bulgaria-Spain football match" 



Neutral 
4 5 



4 
4 
4 
4 
4 



Really 
Want to 
7 



Has watching the "Brighton University Watts Building on Fire: hundreds feared dead" news item changed 
your opinion? How and why? 



Condition B: Same as condition A, only now, 

"Brighton University Watts Building on Fire: hundreds feared dead" is replaced by 
"England football team has to play Bulgaria in the next round". 



